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EXECUTIVE SUMMARY 


As the ongoing national indicator of what American students know and can do, the National 
Assessment of Educational Progress (NAEP) in Reading regularly collects achievement information 
on representative samples of students in grades 4, 8, and 12. Through The Nation’s Report Card, 
the NAEP Reading Assessment reports how well students perform in reading various texts and 
responding to those texts by answering selected-response and constructed-response questions. 

The information NAEP provides about student achievement helps the public, educators, and 
policymakers understand strengths and weaknesses in student performance and make informed 


decisions about education. 


The 2019 NAEP Reading Assessment will measure national, regional, state, and subgroup 
achievement in reading but is not designed to report individual student or school performance. 
The assessment will measure students’ reading comprehension and their ability to apply vocabulary 
knowledge to assist them in comprehending what they read. The reading assessment will use the 
same framework used in 2009. The public will have access to performance results and released ques- 


tions through NAEP reports and websites. 


This document, the Reading Framework for the 2019 National Assessment of Educational Prog- 
ress, presents the conceptual base for and discusses the content of the assessment. It is intended for a 
broad audience. A more detailed technical document, the Reading Assessment and Item Specifications 
for the National Assessment of Educational Progress, is available on the Web. The specifications will 
provide information to guide passage selection, item development, and other aspects of test devel- 
opment. Both the framework and the specifications documents are available to the public at www. 


nagb.org/publications/frameworks.htm. 


The Governing Board, the policymaking body for NAEP, has stated that the NAEP Reading 
Assessment will measure reading comprehension by asking students to read passages written in 
English and to answer questions about what they have read. The framework “shall focus on import- 
ant, measurable indicators of student achievement ... without endorsing or advocating a particular 
instructional approach” (Governing Board 2018). Although broad implications for instruction may 
be inferred from the assessment, NAEP does not specify how reading should be taught; nor does it 


prescribe a particular curricular approach to teaching reading. 
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The NAEP Reading Framework results from the work of many individuals and organizations 
involved in reading and reading education, including researchers, policymakers, educators, and other 
members of the public. Their work was guided by scientifically based literacy research that concep- 


tualizes reading as a dynamic cognitive process as reflected in the following definition of reading. 


Reading is an active and complex process that involves: 


e Understanding written text. 
e Developing and interpreting meaning. 


e Using meaning as appropriate to type of text, purpose, and situation. 


This definition applies to the assessment of reading achievement on NAEP and is not intended 


to be an inclusive definition of reading or reading instruction. 


Text Types 


This framework recognizes that reading behaviors such as recognizing and using features of text, 
making sense of sentences and paragraphs, and comprehending vocabulary occur regardless of text 
type. However, other reading behaviors vary with the type of text encountered by a reader. Thus, 
the NAEP Reading Framework includes two types of texts on the assessment: literary texts, which 
include fiction, literary nonfiction, and poetry; and informational texts, which include exposition, 


argumentation and persuasive text, and procedural text and documents. 


Meaning Vocabulary Assessment 


The NAEP Reading Framework includes a more systematic approach to vocabulary assessment 
than the NAEP Reading Framework used from 1992 through 2007. Vocabulary assessment will 
occur in the context of a passage; that is, vocabulary items will function both as a measure of passage 
comprehension and as a test of readers’ specific knowledge of the word’s meaning as intended by the 
passage author. A sufficient number of vocabulary items at each grade level will provide reliable and 


valid information about students’ vocabulary knowledge. 


Item Design 


The framework includes the following cognitive targets, or behaviors and skills, for items from 
both literary and informational texts: locate/recall, integrate/interpret, and critique/evaluate. 
These cognitive targets illustrate the complex nature of the reading process whereas the correspond- 
ing behaviors highlight the different behaviors elicited by different text types. To measure these 
cognitive skills, students will respond to both selected-response and constructed-response items 
with varying distributions of question type by grade level. Students in grade 4 will spend approx- 
imately half of the assessment time responding to selected-response items and half responding to 


constructed-response items. Students in grades 8 and 12 will spend a greater amount of time on 


constructed-response items. Starting with the 2017 assessment, students engage with both select- 


ed-response and constructed-response items in a digital platform. 


12th Grade NAEP 


In May 2005, the Governing Board adopted a policy regarding NAEP and 12th-grade pre- 
paredness. The policy states that NAEP will pursue assessment and reporting on 12th-grade student 
achievement as it relates to preparedness for postsecondary education and training. This policy 
resulted from recommendations of the Governing Board’s National Commission on NAEP 12th 
Grade Assessment and Reporting in March 2004. Subsequent studies and deliberations by the Gov- 
erning Board took place during 2004 and 2005. 


In reading, the Governing Board adopted minor modifications to the 2009 NAEP Reading 
Framework at grade 12 based on a comprehensive analysis of the framework conducted by Achieve, 
Inc. The current version of the reading framework incorporates these modifications at grade 12 to 
enable NAEP to measure and report on preparedness for postsecondary endeavors. The 2019 NAEP 


Reading Assessment will use the same framework used since 2009. 


Reporting Results 


Results are reported in two ways: as average scores for groups of students on the NAEP 0-500 
scale and as percentages of students who attain each of the three achievement levels (Basic, Profi- 
cient, and Advanced, according to definitions adopted by the Governing Board). NAEP scores are 


always reported at the aggregate level; scores are not produced for individual schools or students. 


Reporting Trend Data 
The Reading Framework for the 2009 National Assessment of Educational Progress replaced the 


framework used first for the 1992 reading assessment and then for subsequent reading assessments 
through 2007. Compared with the previous framework, the 2009 reading framework included more 
emphasis on literary and informational texts, a redefinition of reading cognitive processes, a new 


systematic assessment of vocabulary knowledge, and the addition of poetry to grade 4. 


The 2009 NAEP Reading Report Card included trend data on student reading performance 
from 1992 to 2009. Results from special analyses determined the 2009 reading assessment results 
could be compared with those from earlier assessment years. These special analyses started in 2007 
and included in-depth comparisons of the frameworks and the test questions, as well as an exam- 
ination of how the same students performed on the 2009 assessment and the earlier assessment. A 
summary of these special analyses and an overview of the differences between the previous frame- 
work and the 2009 framework are available on the Web at nces.ed.gov/nationsreportcard/reading/ 
trend_study.asp. 
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The 2019 NAEP Reading Report Card will report trends in student reading performance, 
including achievement results that extend back to1992. 


Sample Items 


Sample passages and items released to the public may be viewed on the NAEP Questions Tool 
at nces.ed.gov/nationsreportcard/itmrlsx/. The Questions Tool also includes performance results, 


scoring rubrics, and student responses to NAEP reading items. 
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PREFACE BY THE NATIONAL ASSESSMENT GOVERNING BOARD 


In a modern society, the ability to read well is the cornerstone of a child’s education. In a mod- 


ern economy, literacy is a prerequisite for a successful life. 


In their early years of schooling, children learn to draw meaning and pleasure from the words on 
a page, which gives them a sense of accomplishment. Throughout the remainder of their schooling, 
reading is the critical skill they use for learning in all parts of the curriculum. For adults, reading is a 
key means to learn and do our jobs; it is also a source of enjoyment and an essential way we connect 
with family, friends, and the world around us. The ability to read critically and analytically is crucial 


for effective participation in America’s democratic society. 


The Reading Framework for the 2019 National Assessment of Educational Progress sets forth the 
design of a test of reading comprehension. The exam requires students to read passages of written 
English text—either literary or informational—and to answer questions about what they have read. 
In some cases, the questions deal with facts in the text or vocabulary. In other cases, a complete 


answer requires a clear analysis or coherent argument supported by sound evidence from the text. 


This is the second reading framework approved by the Governing Board. It replaces the frame- 
work that was used in the National Assessment of Educational Progress (NAEP) from 1992 to 
2007. This new reading framework is the result of extraordinary effort and commitment by hun- 
dreds of people across the country, including some of the nation’s leading figures in reading research, 


assessment, and instruction. 


The new framework incorporates the following key features: 


e Its design is based on current scientific research in reading. In keeping with Governing 
Board policy, it does not advocate a particular approach to instruction, but rather focuses 
on important, measurable indicators of student achievement. 

e The framework’s content and preliminary achievement standards at grade 12 embody read- 
ing and analytical skills the project committees believe are needed for rigorous college-level 
courses and other productive postsecondary endeavors. 

e In preparing the framework, extensive use was made of international reading assessments 
and exemplary state standards. 

e Vocabulary is measured explicitly. Word meanings will be tested in context, and 
enough vocabulary items will be included to report useful information on the extent of 
vocabulary knowledge. 

e Poetry is assessed in grade 4 as well as in grades 8 and 12. Previously, NAEP assessed poetry 
in grades 8 and 12 only. Poetry is a form of text that is rich in meaning and involves a high 


level of abstraction in language and ideas. 
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e Selected-response and constructed-response items (both short and extended) are included 
at all grades. In grades 8 and 12, students will be expected to spend about 60 percent of 
assessment time on constructed-response questions; at grade 4, about 50 percent. 

e Descriptions of reading material to be used in the assessment and target skills to be tested 
are delineated in a series of charts that provide clear guidance to those developing the 
assessment and clear information to the public. 

e Achievement will be reported on an overall cross-grade scale, allowing NAEP to show the 
development of reading skills throughout years of schooling as well as the wide variations 
in particular grades. Clear standards for grade-level expectations will be established. 

e Separate subscales will be reported for literary and informational text, as has been done on 


international reading assessments. 


The Governing Board would like to thank the hundreds of individuals and organizations whose 
time and talents contributed to this reading framework. The framework process was conducted 
through a contract with American Institutes for Research (AIR). Both AIR and another organiza- 
tion, the Education Leaders Council, prepared literature reviews and issues papers, which provided 
different perspectives and served as the basis for extensive discussions by the Reading Framework 
Steering and Planning Committees. These committees, working over a period of 14 months, 
included teachers, reading researchers, local and state policymakers, testing experts, and business and 
public representatives. Many of these individuals have played important roles in other major proj- 
ects, including the National Reading Panel, international reading assessments, the RAND Reading 
Study Group, and the American Diploma Project. 


In addition, the Governing Board convened an independent external review panel comprised 
of eminent reading scholars, authors, and curriculum specialists. Their charge was to conduct an 
in-depth analysis of the framework draft, including its research base and design. These individuals 
played an important role in shaping the framework adopted by the Governing Board. The Govern- 
ing Board also received wide comments on the draft framework through Internet reviews, a public 
forum held in Washington, D.C., and numerous meetings with state and local educators and policy- 


makers across the country. 


We believe the framework will provide a rich and accurate measure of the reading comprehension 
and analytical skills that students need both for their schooling and for their lives. Development of these 
reading skills is the responsibility of all teachers—not only English teachers but also teachers across the 
curriculum—and also involves the expectations of parents and society. The Board hopes that this read- 
ing framework will serve not only as a significant national measure of how well students read, but also as 


a catalyst to improve reading achievement for the benefit of students themselves and for our nation. 
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CHAPTER ONE 


OVERVIEW 


Since 1969, the National Assessment of Educational Progress (NAEP) has been an ongoing 
national indicator of what American students know and can do in major academic subjects, includ- 
ing reading in English. NAEP reading assessments have been administered on a regular schedule to 
students in grades 4, 8, and 12. Since the No Child Left Behind Act of 2001 (NCLB) and continu- 
ing with the Every Student Succeeds Act (ESSA) of 2015, NAEP has assessed reading in grades 4 
and 8 every 2 years. NAEP will also measure reading in grade 12 every 4 years. 


The Reading Framework for the 2019 National Assessment of Educational Progress is one of 
two documents that describe the assessment; it is intended for a general audience and presents the 
conceptual base and content of the assessment. The second document is the Reading Assessment 
and Item Specifications for the National Assessment of Educational Progress and is intended for a 
more technical audience, including the National Center for Education Statistics (NCES) and the 
contractors that will develop the NAEP Reading Assessment. The specifications provide the “test 
blueprint”; that is, information about passage selection, item development, and other aspects of 


test development. 


NAEP Overview 
The National Assessment Governing Board—the policymaking body for NAEP—has defined 


several parameters for the reading assessment. First, the NAEP assessment will measure reading 
comprehension in English. On the assessment, students will be asked to read passages written in 


English and to answer questions about what they have read. 


Second, because this is an assessment of reading comprehension and not listening comprehen- 
sion, NAEP does not allow passages to be read aloud to students as a test accommodation. Third, 
under Governing Board policy, the framework “shall focus on important, measurable indicators 
of student achievement ... without endorsing or advocating a particular instructional approach” 


(Governing Board 2018). Although broad implications for instruction may be inferred from the 


assessment, NAEP does not specify how reading should be taught; nor does it prescribe a particular 


curricular approach to teaching reading. 


Reading passages are selected to be interesting to students nationwide, to represent high-quality 
literary and informational material, and to be free from bias. Students respond to both selected-re- 
sponse and constructed-response items, and in 2019, these item types are presented to students in 
a digital platform. In total, NAEP assessments at grades 4, 8, and 12 are extensive enough to ensure 
that results can be reported validly, but no single student participates in the entire assessment. 
Instead, each student reads approximately two passages and responds to questions about what he 
or she has read. NAEP assessments are administered to random samples of students designed to be 
representative of the nation, different regions of the country, states, and large urban districts. As dis- 
cussed in chapter three, NAEP results are reported for groups of students; no data are reported for 
individual students. Since 1992, states have been able to obtain state-level data on students’ reading 
achievement. Since 2003, a number of large urban school districts were able to obtain data about 
their students’ reading achievement. Results are reported in documents such as the NAEP Read- 
ing Report Card issued following each administration of the reading assessment; through special, 


focused reports; and through electronic means. 


Data are also collected that allow comparison of students’ reading achievement over extended 
periods of time in a separate Long-Term Trend NAEP. These assessments, given at the national level 
only, have been administered in the same form since 1971 and provide the only available measure of 


extended long-term trends in reading achievement. 


Purpose Under NAEP and ESSA Legislation 


The NAEP legislation specifies that NAEP’s purpose is “to provide, in a timely manner, a 
fair and accurate measurement of student academic achievement and reporting of trends in such 
achievement in reading, mathematics, and other subjects[s] ...” (section 303(b)(1), National Assess- 
ment of Educational Progress Reauthorization Act (NAEPRA) of 2002, P.L. 107-279). The NAEP 
reading data will measure national, regional, and subgroup trends in reading achievement but will 


not target the performance of individual students or schools. 


The NAEP Reading Framework is consistent with ESSA legislation, which requires states to 
participate in the NAEP Reading Assessment administered every two years at grades 4 and 8, and 
the resulting data will be widely reported in a timely fashion. Finally, the NAEPRA specifies that 
although the public will have full access to NAEP results and released test questions, NAEP will not 


seek to influence the curriculum or assessments of any state. 
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Definition of Reading for NAEP 


The NAEP Reading Assessment is guided by a definition of reading that reflects scientific 
research, draws on multiple sources, and conceptualizes reading as a dynamic cognitive process. This 
definition applies to the assessment of reading achievement on NAEP and states that reading is an 


active and complex process that involves: 


e Understanding written text. 
e Developing and interpreting meaning. 


e Using meaning as appropriate to type of text, purpose, and situation. 


Terms used in the definition can be further explained as follows: 


Understanding written text: Readers attend to ideas and content in a text by locating and 
recalling information and by making inferences needed for literal comprehension of the text. In 
doing so, readers draw on their fundamental skills for decoding printed words and accessing their 


vocabulary knowledge. 


Developing and interpreting meaning: Readers integrate the sense they have made of the text 
with their knowledge of other texts and with their outside experience. They use increasingly com- 
plex inferencing skills to comprehend information implied by a text. As appropriate, readers revise 


their sense of the text as they encounter additional information or ideas. 


Using meaning: Readers draw on the ideas and information they have acquired from text to 
meet a particular purpose or situational need. The use of text may be as straightforward as knowing 
the time when a train will leave a particular station, or it may involve more complex behaviors such 
as analyzing how an author developed a character’s motivation or evaluating the quality of evidence 


presented in an argument. 


Text: As used in the assessment, the term reflects the breadth of components in typical reading 
materials. Thus, text on the assessment will include literary and informational passages from both 
print and digital sources and may contain noncontinuous text material such as charts. Texts selected 
for inclusion on the assessment represent practical, academic, and other contexts and are drawn 


from grade-appropriate sources spanning the content areas. 


Purpose: Students’ purpose for reading the passages presented on NAEP is determined by the 
assessment context; thus, the influence of purpose on readers’ comprehension is somewhat limited. 
However, the transition to digital-based assessment creates opportunities to introduce more mean- 


ingful purposes such as reading to build and share knowledge or reading to conduct literary analyses. 
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Situation: The situation for reading often determines the way that readers prepare for and 
approach their task. They consider why they are reading (e.g., to study, to relax), how much they 
know about the topic, and other concerns that shape the time they will spend reading. 


Factors That Influence Reading Performance 


Factors related to the text being read and to readers’ backgrounds and experiences influence 
reading performance. For example, understanding the vocabulary, concepts, and structural ele- 
ments of the text contributes to readers’ successful comprehension. Comprehension is also affected 
by readers’ background knowledge and by the context of the reading experience. The background 
knowledge that students bring to the NAEP Reading Assessment differs widely. To accommodate 
these differences, passages will span diverse areas and topics and will be as engaging as possible to the 


full range of students in the grades assessed. 


The purpose for reading also influences performance. In the case of the NAEP Reading Assess- 
ment, purpose is determined by the assessment context; thus, the influence of purpose on readers’ 
comprehension is somewhat limited. For this reason, the definition of reading presented earlier 
should be considered as a guide for the NAEP Reading Assessment, not as an inclusive definition of 
reading. The definition pertains to how NAEP defines reading for the purpose of this assessment. It 
does not address the issue of how students should be taught to read. 


Text comprehension is influenced by readers’ ability to apply the essential components of 
reading: phonemic awareness, phonics knowledge, fluency, and understanding of word meanings or 
vocabulary. Without these foundational skills, comprehension will not occur. By grade 4, when the 
NAEP Reading Assessment is first administered, students should have a well-developed understand- 
ing of how sounds are represented alphabetically and should have had sufficient practice in reading 
to achieve fluency with different kinds of texts (National Research Council 1998). Because NAEP 
tests at grades 4, 8, and 12, the assessment focuses on students’ reading comprehension, not their 


foundational skills related to alphabetic knowledge.’ 


As discussed further in chapter two, the association between vocabulary knowledge and com- 
prehension is strong; students who know the meanings of many words and who also can use the 
context of what they read to figure out the meanings of unfamiliar words are better comprehenders 
than those who lack these attributes (National Reading Panel 2000a). In the NAEP Reading Assess- 
ment, vocabulary will be assessed systematically through carefully developed items that measure 


students’ ability to derive the meanings of words within the context of the passages they read. 


' NAEP investigated the relationship between oral fluency and reading comprehension in two special studies in 1992 
and 2002. 


Nature of Reading Behaviors 


Reading is an active and complex process that involves multiple different behaviors. Readers 
often begin by forming an overview of text and then search for information to which they must pay 
particular attention. Following this initial overview, readers progress with different levels of interac- 
tion with text, including interpreting and evaluating what they read. By drawing on previous reading 
experiences and prior knowledge, they form hypotheses about what the text will communicate and 
revise their initial ideas and their knowledge base as their reading continues. Readers continuously 
acquire new understandings and integrate these into their ongoing process of building comprehen- 
sion. Good readers monitor their understanding of text, recognize when text is not making sense, 
and employ a range of strategies to enhance their comprehension. Good readers also evaluate the 
qualities of text, and these evaluations can affect whether a text is remembered or has an impact 
on readers’ knowledge, attitudes, or behaviors (Pressley and Afflerbach 1995; Ruddell and Unrau 
1994). Depending on the situation and purpose for reading, good readers can use the ideas and 
information they acquire from text to, for example, expand their thinking about a topic, perform a 


specific task, or draw conclusions or make generalizations about what they have read. 


Definitions of Reading That Have Informed Framework Development 


The definition of reading for the NAEP Reading Assessment is derived from several sources and 
grounded in scientific research on reading. Among the sources are the Elementary and Secondary 
Education Act (as amended by NCLB and ESSA), several important research reports on reading, 
and the definitions of reading that guide the development of international reading tests. Each 
source has contributed important ideas to the definition of reading used for the NAEP Reading 


Assessment. 


NCLB and ESSA posit that reading has five essential components: phonemic awareness, pho- 
nics, reading fluency, vocabulary, and comprehension. The NAEP Reading Assessment measures 
students’ meaning vocabulary and comprehension. To demonstrate comprehension of what they 
read, students use their phonemic awareness and knowledge of phonics. Their ability to read the pas- 
sages and test questions with minimal effort reflects their fluency. Students draw on their vocabulary 
knowledge throughout the assessment, and specific items ask about carefully selected target words 


in each reading passage. 


The National Reading Panel (NRP) (National Institute of Child Health and Human Develop- 
ment 2000), a congressionally mandated commission, conducted an extensive, evidence-based study 
of research literature on reading acquisition, reading growth, and other relevant topics. The NRP 
report was an important foundation for NCLB, and later ESSA, highlighting the importance of 


alphabetics (phonemic awareness and phonics), fluency, vocabulary, and comprehension. 
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Three important definitions of reading influenced the development of the definition of reading 
for the NAEP Reading Assessment. The first comes from Reading for Understanding: Toward an 
R&D Program in Reading Comprehension (RAND Reading Study Group 2002), frequently referred 
to as the RAND Report. This report was prepared by the RAND Reading Study Group under the 
auspices of the Office of Educational Research and Improvement of the U.S. Department of Educa- 
tion. Guiding the work of the study group was the following definition of reading: 


Reading comprehension {is} the process of simultaneously extracting and constructing 
meaning through interaction and involvement with written language. It consists of 


three elements: the reader, the text, and the activity or purpose for reading (p. 11). 


The second important definition was the foundation for item development for the Progress in 
International Reading Literacy Study (PIRLS) (Campbell et al. 2001). PIRLS was first adminis- 
tered to 9-year-old students in 35 countries in 2001. PIRLS defines reading literacy as: 


The ability to understand and use those written forms required by society and/or val- 
ued by the individual. Young readers can construct meaning from a variety of texts. 


They read to learn, to participate in communities of readers, and for enjoyment (p. 3). 


The Programme for International Student Assessment (PISA) (OECD 2000) represents an 
international collaborative effort to assess what 15-year-old students know and can do in reading, 


mathematics, and science. PISA defines reading literacy as: 


Understanding, using, and reflecting on written texts, in order to achieve one’s goals, 


to develop one’s knowledge and potential, and to participate in society (p. 18). 


The RAND Report, PIRLS, and PISA offer support to the definition of reading advocated in 
the NAEP Reading Framework. All three stress that reading is an active, complex, and multidimen- 


sional process undertaken for many different purposes. 


Overview of NAEP Reading Assessment 


This reading assessment will use the same framework used in 2009. The NAEP Reading Assess- 
ment will include two distinct types of text at grades 4, 8, and 12. Doing so will allow the develop- 
ment of items that measure students’ comprehension of the different kinds of text they encounter 
in their school and out-of-school reading experiences. The reasons for including literary and infor- 
mational text are presented next, followed by explanations of the characteristics of each text type 
included on the assessment. The NAEP Reading Assessment will also include items that measure 


students’ ability to apply their knowledge of vocabulary as an aid in their comprehension process. 


NAEP assesses reading skills that students use in all subject areas and in their out-of-school and 
recreational reading. By design, many NAEP passages require interpretive and critical skills usually 
taught as part of the English curriculum. However, NAEP is an assessment of varied reading skills, 
not a comprehensive assessment of literary study. The development of the broad range of skills that 
the nation’s students need to read successfully in both literary and informational texts is the respon- 


sibility of teachers across the curriculum, as well as of parents and the community. 


Commonalities in Reading Behaviors Across Text Types 


The framework recognizes that even though there are substantial differences in reading behay- 
iors for different text types, there are also great similarities. Regardless of the type of text, the reader 
must access the words in the text, recognize and use the structure of the text, make sense of sen- 
tences and paragraphs, and comprehend what has been read. Equally, vocabulary is a critical element 


in comprehending any kind of text. 


Text Characteristics: Literary and Informational Texts 


Research on the nature of text and on reading processes has suggested that the characteristics of 
literary and informational text differ dramatically. For the most part, the research literature suggests 
that readers pay attention to different aspects of text as they seek to comprehend different text types 
(Pearson and Camperell 1994; Pressley 2000; Purves 1973). Additionally, the PIRLS report shows 
that students in the United States scored higher on the Literary Subscale (at 550) than on the Infor- 
mational Subscale (at 533), further substantiating the difference in the strategies needed for the two 
text types (OECD 2000). An earlier international study reported that patterns of student responses 
to literature were influenced by the nature of the selections they were given to read. Different lit- 
erary samples elicited different responses from students with some consistency across cultures and 
school systems (Purves 1973). Drawing on this extensive research base, the NAEP Reading Frame- 
work includes two major types of text: literary and informational. Well-crafted nonfiction work 
with strong literary characteristics will be classified as literary text and documents such as tables, 


graphs, or charts will be included in the informational category. 


Literary and informational texts for the NAEP Reading Assessment are separated for two pri- 
mary reasons: the structural differences that mark the text types and the purposes for which stu- 
dents read different texts. Exhibits 3 and 4 in chapter two present details about the kinds of literary 
and informational texts to be included on the NAEP Reading Assessment and about the features of 


these texts for which items will be written. 
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Structural Differences in Text 


Literary and informational texts are marked by distinct structural characteristics that readers 
rely on as they seek to understand what they read (Goldman and Rakestraw 2000). For example, 
research on literary text (Graesser, Golding, and Long 1991) has pointed out that stories and novels 
are characterized by a coherent text structure known as story grammars. Research on informational 
or expository text (Kobayashi 2002) has indicated that such texts possess distinct organizational 
patterns, such as sequence or comparison and contrast, designed to help readers organize their 
emerging sense of what the text is communicating. These structures are distinct from story gram- 
mars. The nature of texts affects comprehension, and different text types must be read in different 
ways (Pearson and Camperell 1994). Good readers adjust their reading behaviors to accommodate 


the kinds of text they are reading. 


Purposes for Reading 


A second reason for separating text types is that readers often read literary and informational 
texts for different purposes. The definition of reading that guides the NAEP Reading Assessment 
specifically states that readers read for different purposes, which are often reflected in their selection 
of literary or informational texts. The purpose set for reading a text often determines how a stu- 
dent reads that text. Literary texts, such as stories, drama, essays, or poetry, are frequently read for 
pleasure or for new perspectives on time, place, human nature, or feelings; they are often read from 
beginning to end. The ultimate utility of informational text is determined by how well it conveys 
information or ideas. These differences in reading purpose are, of course, permeable. For example, 
well-crafted informational text is often read for appreciation and enjoyment, in addition to the 


information that the text can provide. 


Features That Distinguish Text Types 


Several features distinguish literary and informational texts. Skilled writers understand that dif- 
ferent kinds of text need different structural patterns, and good readers are able to use these specific 


text features as aids in comprehension. 


Literary Texts 


The NAEP Reading Assessment will present reading passages (i-e., stimulus material) drawn 


from three categories of literary text: 


e Fiction 
e Literary nonfiction, such as essays, speeches, and autobiographies or biographies 


e Poetry 
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The structural patterns of fiction (i.e., short stories and novels) have been studied extensively. 
Although many researchers have suggested different ways to name the elements of a story (Stein and 
Glenn 1979), there is general agreement that a story consists of the following components: the set- 
ting or settings; a simple or complex plot consisting of a series of episodes and delineating a problem 
to be solved; the problem or conflict, which requires characters to change, revise plans, or face chal- 
lenges as they move toward resolution; and a reaction that expresses the protagonist’s feelings about 
his or her goal attainment or relates to the broader consequences of the conclusion of the story. This 
structure is often referred to as a story grammar. Characters populate each story, in major or minor 


roles; themes or major ideas are stated either implicitly or explicitly. 


Works of literary nonfiction such as biographies, essays, and speeches employ distinct, varied 
structural patterns and literary features to reflect their purpose and audience. These works may not 
only present information and ideas, but they also employ distinctly literary elements and devices to 
communicate their message and to make their content more accessible to readers. Biographies and 
autobiographies, for example, usually follow a structure that in many ways mirrors the story struc- 
ture of fictional works, and they may employ literary devices, but they also present information. 
Literary essays and speeches may be structured differently but also draw on literary devices. The Get- 
tysburg Address, for example, might be viewed simply as an argumentative text or as a dedication or 
a eulogy, but it is more appropriately viewed as a sophisticated literary text. Readers approach texts 
of this type not only to gain enjoyment and information, but also to learn and to appreciate the 


specific craft behind authors’ choices of words, phrases, and structural elements. 


Like fiction and literary nonfiction, poetry demonstrates specific text characteristics, but these 
characteristics are different from those found in continuous prose (Hanauer forthcoming). Some 
poetry possesses very rhythmic or metrical patterns, and some is written as free verse without a regu- 
lar line pattern. Poetry is a highly imaginative form of communication in that poets try to compress 
their thoughts in fewer words than would be used in ordinary discourse or in prose (Frye 1964). 
Because the language is often brief and concise, poems employ picturesque and evocative words as 
well as similes, metaphors, personification, imagery, and other devices that convey the symbolic 
nature of the ideas, emotions, and actions being expressed. Poetry often involves a high level of 
abstraction in language and ideas, and requires specific critical thinking skills not found in other 
types of literary works. For these reasons, it is important that NAEP include poetry at grades 4, 8, 
and 12. 
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Informational Texts 


For the NAEP Reading Assessment, informational texts will be classified into three 


broad categories: 


e Exposition 
e Argumentation and persuasive text 


e Procedural text and documents 


Informational text, specifically exposition, argumentation, and persuasive text, does not have 
a single, identifiable structure. Rather, different types of informational text exhibit distinct struc- 
tural features. The most common structural patterns for continuous expository, argumentative, and 
persuasive text can be summarized as follows (Bovair and Kieras 1991; Meyer 1975; Goldman and 
Rakestraw 2000; Kobayashi 2002): 


Description: A descriptive text structure presents a topic with attributes, specifics, or setting 


information that describe that topic. 
Sequence: Ideas are grouped on the basis of order or time. 


Causation: The text presents causal or cause-and-effect relationships among the ideas presented 


in the text. 


Problem/Solution: The main ideas are organized into two parts: a problem and a subsequent 


solution that responds to the problem, or a question and an answer that responds to the question. 


Comparison: Ideas are related to one another on the basis of similarities and differences. The 


text presents ideas organized to compare, to contrast, or to provide an alternative perspective. 


Expository text, argumentation, and persuasive text often contain pictures, charts, tables, and 
other graphic elements that augment text and contribute to its meaning. Ancillary aids such as head- 
ings, bolded text, or bulleted lists emphasize specific components of the text to reinforce authors’ 
messages. Literary texts differ in that illustrations, pictures, or other nonprint elements (when pres- 


ent) may aid readers in understanding the text but are not usually critical for comprehension. 


The first kind of informational text on the NAEP Reading Assessment, exposition, presents 
information, provides explanations and definitions, and compares and contrasts. Textbooks, news 
stories, and informational trade books are examples of expository text. Texts classified as argumen- 
tation or persuasive text accomplish many of these same goals but can be distinguished by their 


particular purpose and by the features that authors select to accomplish their goals for writing. 
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The second category of informational text includes argumentation and persuasive text (Driver, 
Newton, and Osborne 2000; Osborne 2002; Wineburg 1991). Argumentation seeks to influence 
through appeals that direct readers to specific goals or try to win them to specific beliefs. Authors of 
persuasive writing must establish their credibility and authority if their writing is to be successful. 


Examples of persuasive text are political speeches, editorials, and advertisements. 


The third type of informational text is often categorized as procedural texts or documents (Kirsch 
and Mosenthal 1990; Mosenthal 1996; Mosenthal 1998). Procedural texts convey information in 
the form of directions for accomplishing a task. A distinguishing characteristic of such text is that 
it is composed of discrete steps to be performed in a strict sequence with an implicit end product or 
goal. After reading the text, the reader should be able to reach a goal or complete a product. Exam- 
ples include (but are not limited to) manuals and product support materials, directions for art activ- 
ities and hobbies, and so on. Procedural texts may include information arranged in graphs, charts, or 


maps, in addition to prose. 


Document texts in a variety of forms will also be represented on the NAEP Reading Assessment. 
Documents include graphical representations, often as multimedia elements that require readers to 
draw on information presented as short continuous prose and also as columns, matrices, or other 
formats. Document structures can be simple or complex, and can present information in a straight- 
forward way as in a simple list or pie graph with clearly delineated elements or embed or nest infor- 


mation within a document’s structure. 


Documents are used frequently in schools and in society. Textbooks often include graphs, tables, 
and illustrations to accompany and expand on traditional text. Forms are also common (such as 
applications), as are procedural texts (such as manuals and directions). Documents have implicit 
procedures embedded within them. Often, readers must cycle through the document or the set of 
procedures to gain needed information or to answer specific questions. For example, instructions 


suggest the manner in which an application is to be completed. 


Informational text will be included at all levels of the NAEP Reading Assessment. Documents 
embedded in text will be used at grades 4 and 8; stand-alone documents that provide enough infor- 
mation to support item development may be used at grade 12. Chapter two describes the criteria for 


evaluating examples and noncontinuous text and documents for inclusion. 


Percentage of Passages by Text Type and Grade 


Exhibit 1 shows the distribution of literary and informational passages on the assessment. The 
percentage listed for literary texts encompasses all three categories of text: fiction, literary nonfic- 


tion, and poetry. The percentage for informational text likewise includes exposition, argumentation 
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and persuasive texts, and procedural texts and documents. The Specifications for the NAEP Reading 


Assessment will detail how these percentages are to be distributed across grades 4, 8, and 12. 


The distribution reflects the kinds of texts that students read across the curriculum as they prog- 
ress through elementary, middle, and high school (Alexander and Jetton 2000). It further reflects 
the distribution of text types on many state reading tests designed to reflect what students read 


across the curriculum. 


Exhibit 1. Percentage distribution of literary and informational passages 


Grade Literary Informational 
zs 50 50 
8 45 55 
12 30 70 


Mixed Texts 


Many of the texts that convey information have been termed mixed texts (Alexander and Jetton 
2000). This type of text is common in classroom reading as students are introduced to informational 
texts as a genre distinct from the stories common in lower grades (Duke 2000; Leu and Kinzer 
2000). Examples include historical or scientific accounts presented in quasi-narrative form but used 
to communicate information. Their literary qualities (for example, literary elements and devices) 


will determine their classification as literary or informational. 


Multiple Texts 


A common task for readers at all grades is integrating information across a set of texts. It is often 
the case that readers have multiple questions for which they need or want answers. A single text 
may answer some questions incompletely, or a single text might contain answers for only a portion 
of the questions a reader has. The solution is to use other texts to find additional information. In 
consulting multiple texts, readers must engage in all the processes to read individual texts, and they 
must also engage in other processes to compare those texts on multiple dimensions and decide on 
their accuracy, bias, and credibility. These skills need to be assessed to see how well students can read 
and comprehend texts that contain different information, reach different conclusions about the 
same material, or have different levels of credibility. Continuing the use of intertextual passage sets 
as part of the NAEP Reading Assessment approximates the authentic task of reading and comparing 


multiple texts. 
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The transition to digital administration also provides opportunities to expand the purposes 
for which we ask students to read on the assessment and to use a wider range of texts, including 
those coming from digital sources that may involve dynamic features such as video, animation, 


or hyperlinks. 


Vocabulary Assessment on the NAEP Reading Assessment 


The Governing Board has endorsed the idea of measuring students’ vocabulary as part of the 
reading assessment and supports an approach that assesses vocabulary in the context of the reading 
passages. The goal of vocabulary assessment will be to measure students’ meaning vocabulary, which 


can be defined as follows: 


Meaning vocabulary is the application of one’s understanding of word meanings to 


passage comprehension. 


The proposed method of assessing meaning vocabulary on the NAEP Reading Assessment 
assumes that the ability to gain a sense of the meaning of all or most words in a passage, especially 
those words that convey important information linked to central ideas of the passage, is a necessary 
condition for comprehension. NAEP meaning vocabulary items will target words already present 
in the NAEP reading comprehension passages. Candidate words must convey important meaning 
linked to the central idea(s) of the passage; comprehension is likely to be disrupted if the meaning of 
the test word was not known. It is anticipated that each passage will have approximately two vocab- 


ulary items. The vocabulary assessment is explained in detail in chapter two. 


Assessing Students With Special Needs 


The NAEP Reading Assessment is designed to measure the academic achievement of all test tak- 
ers at a given grade level; hence, students with disabilities and English language learners are included 
in the assessment sample. The assessment is administered to English language learners and students 
with disabilities who, based on inclusion criteria provided by NAEP, are capable of participating. 
Special care is taken in designing and developing the assessment to ensure that these students, along 
with all others, find the passages and items accessible. For example, passages that might require spe- 
cific background or experiential knowledge for comprehension are not included in the assessment. 


Items are written in plain language without jargon or complex syntactical structures. 


Some students may need accommodations to be able to participate in the NAEP Reading 
Assessment. NAEP attempts to provide accommodations to students that match the way in which 
they are tested in school as long as those accommodations do not alter the construct being mea- 


sured. For example, large-print versions are made available for students with visual impairments; 
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students with disabilities may be given one-on-one or small-group testing situations or extended 
time to complete the assessment. Some students, for example those who are learning English, may 
have the test directions (but not the passages or items) read orally to them by an assessment admin- 
istrator or as part of a digital platform. Other students may benefit from having a trained aide tran- 
scribe dictated responses for them. Accommodations may be provided in combination, for example, 


extended testing time and individual administration of the assessment. 


Comparison of 1992-2007 NAEP Reading Framework and 2009-2019 NAEP 
Reading Framework 


The framework for the 2009-2019 NAEP Reading Assessment replaces a framework devel- 
oped for the 1992 assessment. The previous framework was refined during its use to reflect more 
clearly the goal of precisely measuring students’ reading skills and strategies and was reissued for the 
2003 assessment. The new framework honors many aspects of the previous one, but also introduces 
some changes that can lead to better measurement and more precise reporting of assessment results. 


Important changes featured in the new NAEP Reading Framework follow: 


e An assessment design based on current scientific reading research. 

e Use of international reading assessments to inform the NAEP framework. 

e More focused measurement of vocabulary. 

e Measurement of reading behaviors (cognitive targets) in a more objective manner. 
e Distinction of cognitive targets relevant to literary and informational text. 

e Use of expert judgment, augmented by readability formulas, for passage selection. 
e Testing of poetry at grade 4 in addition to grades 8 and 12. 


e Special study of vocabulary to inform development of the assessment. 


Key similarities and differences between the two frameworks are presented in exhibit 2. Chapter 
two explains the proposed content and design of the assessment. The content and cognitive targets, 
as operationalized to reflect the definition of reading presented earlier in chapter one, will yield 
passages and items that reflect the complex interaction of the reader, the text, and the context of 
the assessment. 
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Exhibit 2. Similarities and differences: 
1992-2007 and 2009-2019 NAEP reading frameworks 


1992-2007 NAEP 


2009-2019 NAEP 


Reading Framework Reading Framework 
Content of assessment: | Contexts for reading: e —_ Literary text e = Informational text 
e e —_ Literary e For literary e.— Fiction e Exposition 
8 e — Informational experience e Literary e  Argumentation 
& e Document e For information nonfiction and persuasive text 
e To perform task e Poetry e Procedural text 
and documents 
g Stances/aspects of reading: Cognitive targets distinguished by text type 
é e Forming general understanding. beck Integrate/ Critique/ 
g e Developing interpretation. recall interpret evaluate 
fe e Making reader/text connections. 
2 e Examining content and structure. 
2 
1S) 
se Vocabulary as a target of item development, with no _ | Systematic approach to vocabulary assessment with 
@ | information reported on students’ use of vocabulary | potential for a vocabulary subscore. 
E knowledge in comprehending what they read. 
5 
S 
p» | Poetry included as stimulus material at grades 8 Poetry included as stimulus material at all grades. 
g and 12. 
Ay 


Passage 
Source 


Use of intact, authentic stimulus material. 


Use of authentic stimulus material plus some 


flexibility in excerpting stimulus material. 


Passage 
Length 


Grade 4: 250-800 words 
Grade 8: 400—1,000 words 
Grade 12: 500-1,500 words 


Grade 4: 200-800 words 
Grade 8: 400-1,000 words 
Grade 12: 500-1,500 words 


Passage 
Selection 


Item Type 


Expert judgment as criterion for passage selection. 


Selected-response and constructed-response items 


included at all grades. 


Expert judgment and use of at least two research- 


based readability formulas for passage selection. 


Selected-response and constructed-response items 


included at all grades. 
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CHAPTER TWO 


CONTENT AND DESIGN OF NAEP IN READING 


This chapter presents the content and design of the NAEP Reading Assessment. Key sections of 


the chapter are as follow: 


e Texts to be included on the NAEP Reading Assessment 

e Characteristics of texts selected for inclusion on the NAEP Reading Assessment 
e Literary text 

e Informational text 

e Vocabulary on the NAEP Reading Assessment 

e Cognitive targets for the NAEP Reading Assessment 

e Item types on the NAEP Reading Assessment 


Texts on the NAEP Reading Assessment to Be Included 


The National Assessment of Educational Progress in Reading will assess students’ comprehen- 


sion of literary and informational passages. Within these passages, vocabulary will also be assessed. 


Chapter one presented the rationale for including literary and informational text on the NAEP 
Reading Assessment, and this chapter begins by describing the text structures and features and 


aspects of author’s craft about which items will be developed. 


The matrices in exhibits 3 and 4 show the kinds of literary and informational texts that will be 
sampled at grades 4, 8, and 12, along with the text structures and literary devices or elements of 


author’s craft about which items may be developed. 


The matrices are designed to show the following aspects of literary and informational text: 


e Genres and types of text to be assessed. 
e Text structures and features about which items may be asked. 


e Aspects of author’s craft about which items may be asked. 


Types of text refers to the idealized norms of a genre (Fludernik 2000), not the source of the 


stimulus material per se. 
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Text structures and features define the organization and elements within the text. The organiza- 
tion and elements refer to the ways ideas are arranged and connected to one another. Features refer to 


visual and structural elements that support and enhance the reader’s ability to understand the text. 


Author's craft pertains to the specific techniques that an author chooses to relay an intended 


message. 


Entries listed within each cell of the matrices should be construed as neither definitive nor 
inclusive of all text structures and features or techniques of author's craft. However, it is important 
to delineate the type of text to be used in reading comprehension tests (Kobayashi 2002; Wixson 
and Peters 1987). Understanding the range of text types for inclusion in the NAEP Reading Assess- 
ment illuminates the complex nature of reading comprehension passages and the accompanying 
questions. Items will assess students’ application of knowledge about text types, text features and 
structures, and author's craft, not their recognition of specific terminology in isolation. The designa- 
tion of entries in the matrices by grade level reflects the levels at which these components of text are 
presented in state English language arts standards. They have further been confirmed by experienced 


teachers and teacher educators. 


Literary Text 


The literary text matrix shown in exhibit 3 outlines the common forms of continuous prose and 
poetry that may be included. The matrix is divided into three sections (fiction, literary nonfiction, 
and poetry) and provides information on the aspects of text about which items will be developed. 


Successively more complex text forms are added at each level.” 


> A detailed explanation of the literary and informational text matrices will be provided in the Specifications for the 
NAEP Reading Assessment. 
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Exhibit 3. Literary text matrix: Fiction 


Genre/Type of Text Text Structures and Features Author’s Craft 
e Adventure stories e Themes Diction and word choice 
e — Historical fiction e = Morals e Dialogue 
e Contemporary realistic e Lessons e __Exaggeration 
fiction Organization e _- Figurative language 
< e ~—- Folktales e Plot: sequence of events e Symbolism 
Zl e Legends e «© Conflict e Simile and metaphor 
G e Fables e ~—- Solution 
e ~—_ Tall tales e ~—_- Resolution 
e Myths Elements 
e Fantasy e — Setting 
e Characterization 
e Science fiction Organization e Mood 
e Parallel plots e Imagery 
e Circular plots e — Flashback 
g s e —_ Elements e Foreshadowing 
3 a e Point of view e _ Personification 
ea & e Contradictions 
e —_ Internal vs. 
external conflict 
Plus increasingly complex Plus increasingly complex Plus increasingly complex 
application of grade 4 application of grade 4 application of grade 4 
e = Satire Organization e Dramatic irony 
e Parody e Differentiation of plot e Character foils 
¢ Allegory structures for different é- Comiczelief 
a e Monologue purposes and audiences e Unconventional 
ce Elements use of language 
¥ e Interior monologue 
o e Unreliable narrators 
e Multiple points of view 
Plus increasingly complex Plus increasingly complex Plus increasingly complex 
application of grades 4 and 8 application of grades 4 and 8 application of grades 4 and 8 


Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 
features or techniques of author's craft. 


Fiction 


As suggested in the matrix, students in elementary and middle schools read many different kinds 
of stories for enrichment and enjoyment. These texts are representative of the developing conceptual 
understandings formed by students during this period. At grades 8 and 12, more complex genres of 


fiction are common including satires, parodies, science fiction, and allegories. For purposes of the 
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NAEP Reading Assessment, these genres may be either intact passages or passages excerpted from 
longer genres such as novels. Material excerpted from longer pieces will be carefully analyzed to 


ensure that it has the structural integrity and cohesion necessary to sustain item development. 


The matrix also shows the aspects of text structures and features and author’s craft that may be 
assessed. These components, as well as the purposes for reading, become increasingly complex and 
sophisticated as students move through the elementary, middle, and high school grades. For exam- 
ple, themes may be more abstract; plots may involve internal or external conflicts; characterization 
may develop with antagonists, protagonists, and narrators with intertwined motives, beliefs, traits, 
and attitudes; the theme and setting may be more integral to one another; the plot may consist of 
a series of rising and falling actions within episodes; and the point of view or vantage point chosen 
by the author to reveal ideas, characters, or actions becomes more sophisticated, often including a 


shifting point of view or multiple points of view. 


Authors select from a range of stylistic devices to enhance their presentation. In the matrix, 
these are referred to as author’s craft. At grade 4, author’s craft includes figurative language such as 
symbolism, simile, metaphor, diction and word choice, dialogue, and exaggeration. More abstract 
elements, such as flashback and imagery, are part of author’s craft at grade 8 in addition to more 
complex applications of the types of author’s craft listed for grade 4. Fictional passages for grade 12 
are complex and may include the following literary devices of dramatic irony, character foils, comic 


relief, and unconventional use of language in addition to the devices listed at grades 4 and 8. 


| 20 


Exhibit 3 (continued). Literary text matrix: Literary nonfiction 


Genre/Type of Text Text Structures and Features Author’s Craft 
e Personal essay Organization e __ Diction and word choice 
e Autobiographical and e ~=—- Description e Use of exposition, action, 
biographical sketches e = Cause and effect or dialogue to introduce 
e Comparison characters 
oR * Chronology e —_ Exaggeration 
8 Elements e _- Figurative language 
fo 
ie) e ~—— Point of view e Symbolism 
e Themes or central ideas e Simile and metaphor 
% e Supporting ideas 
2 e Logical connections 
7 
3) oe 
a e _ Transitions 
=| 
9 Character sketch e Voice 
e e Memoir e Tone 
3 = ie Speech e Imagery 
= vo , 
— 3 e Metaphoric language 
oO e —_ Irony 
Plus increasingly complex Increasingly complex application | Plus increasingly complex 
application of grade 4 of grade 4 application of grade 4 
e Classical essay ° Denotation 
N ‘ 
= e Connotation 
= 
& Plus increasingly complex Increasingly complex application | Plus increasingly complex 
application of grades 4 and 8 of grade 4 application of grades 4 and 8 


Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 


features or techniques of author's craft. 
Literary Nonfiction 


The second type of literary text is literary nonfiction, which may include elements of narration 
and exposition and is often referred to as mixed text (Alexander and Jetton 2000). Literary nonfic- 
tion is an example of mixed text because it uses literary techniques usually associated with fiction or 
poetry and also presents information or factual material. Stylistically, it frequently blends literary 
elements and devices with factual information with the dual purpose of informing and offering read- 
ing satisfaction. Text types for literary nonfiction at grade 4 include autobiographical and biographi- 
cal sketches and personal essays. At grade 8, additional forms of literary nonfiction include character 
sketches, memoirs, and speeches. Classical essays are introduced as literary nonfiction at grade 12. 
Unlike texts that can be categorized as informational because of their sequential, chronological, 


or causal structure, literary nonfiction uses a story-like structure. Classical essays may interweave 
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personal examples and ideas with factual information to attain their purpose of explaining, present- 


ing a perspective, or describing a situation or event. 


Literary nonfiction selected for inclusion on NAEP will conform to the highest standards of 
literary quality. Literary nonfiction combines structures from both literary and informational texts. 
At grade 4, text structures and features in literary nonfiction include description, cause and effect, 
comparison, chronology, point of view, themes and central ideas, and supporting ideas. At grades 8 
and 12, increasingly complex structures listed above are noted in literary nonfiction. Text features 


such as logical connective devices and transitional devices are listed at grade 4. 


A range of literary devices and techniques termed author’s craft is present in literary nonfiction. 
Examples of author’s craft at grade 4 include diction and word choice, various ways to introduce 
characters, exaggeration, and figurative language. At grade 8, increasingly complex techniques are 
listed for author’s craft: voice, tone, imagery, metaphoric language, and irony. Denotation and con- 
notation are listed at grade 12 for author's craft. Grades 8 and 12 will include more complex forms 


of the text structures and features and author’s craft listed at grade 4. 
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Exhibit 3 (continued). Literary text matrix: Poetry 


Text Structures and 


Poetry 


Genre/Type of Text Author’s Craft 
Features 
e = Narrative poem Organization e _ Diction and word choice 
e —- Verse (including the decision to omit 


e = Lyrical poem 
e Humorous poem 


e Free verse 


e = Stanza 
Text features 
e Repetition 


e Omission 


words that may leave the reader 
with much to infer) 

Choice of different forms 

of poetry to accomplish 


Plus increasingly complex 


application of grades 4 and 8 


Plus increasingly complex 


application of grades 4 and 8 


e * Dialogue different purposes 
3 e —_ Line organization ¢ — Exaggeration 
3 : Wes e Use ofimagery to 
Elements provide detail 
“ Rhyme scheme e Figurative language 
© Rhythm e Simile 
5 Missa e Metaphor 
I 
e Themes and intent : ee, 
e Alliteration 
e Onomatopoeia 
e Ode Elements e _ Figurative language 
e Song (including ballad) e Abstract theme e Symbolism 
i e = Epic e Rhythm patterns e Personification 
~~ 
g e —— Point of view 
ie) 
Plus increasingly complex appli- | Plus increasingly complex Plus increasingly complex application 
cation of grade 4 application of grade 4 of grade 4 
e Sonnet Elements e  Denotation 
e = Elegy e Complex themes e Connotation 
e = Multiple points of view e Irony 
S e Interior monologue e ‘Tone 
S e — Soliloquy e Complex symbolism 
& . 
S e — Iambic pentameter e — Extended metaphor and 


analogy 


Plus increasingly complex application 


of grades 4 and 8 


Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 


features or techniques of author's craft. 


Poetry 


The third type of literary text included in the NAEP Reading Assessment is poetry. Like fiction, 


poetry has distinctive forms, functions, and structures further guided by literary structures and 


textual features. The matrix lays out the kinds of poetry that students encounter at different grade 
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levels. Thus, basic poetic forms at grade 4 are narrative, lyrical, and humorous poems and free verse. 
Additionally at grade 8, odes, songs, and epics are included in the matrix for possible item develop- 
ment. More complex poetic forms are included at grade 12, such as sonnets and elegies. It is possible 
that two poems may be used together in intertextual item sets to allow students to perform complex 
reading tasks, such as comparing thematic treatment in two poems or contrasting two poets’ choices 
of literary devices. 


Readers use the structure of poetry to aid in comprehension. Poetic structures range from simple 
to complex. Students at grade 4 can be expected to be familiar with simple organizational patterns 
such as verse and stanza along with the basic elements of rhyme scheme, rhythm, mood, and themes 
and intent. At grades 8 and 12, increasingly complex poetic organizational patterns and elements 
will be included. Students will also be expected to understand the use of white space as a structural 


feature of poetry. 


Understanding a poet’s choices also aids in understanding poetry. Language choice is of partic- 
ular importance because the meaning in poetry is distilled to as few words as possible. Poets choose 
from among a range of rhetorical structures and figurative language, using, for example, repetition, 
dialogue, line organization and shape, patterns, and many forms of figurative language. Increasingly 
complex applications of figurative language, rhetorical devices, and complex poetry arrangements 
are included at grades 8 and 12. 


Informational Text 


As stated in chapter one, informational text on the NAEP Reading Assessment will be of three 
types: exposition, argumentation or persuasive text, and procedural text or documents. Exhibit 4 
presents the ways informational text will be assessed at grades 4, 8, and 12. The matrix consists of 


three parts, each of which is accompanied by explanatory text. 
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Exhibit 4. Informational text matrix: Exposition 


Genre/Type of Text Text Structures and Features Author’s Craft 
e Informational trade book e Organization e Transitional words 
e Textbook e Description e Signal words 
e News article e Sequence (e.g., enumera- e Voice 
e —_- Feature article tion, chronology) e _ Figurative language and 
e — Encyclopedia entry e = Cause and effect rhetorical structures 
« Bookreview e Problem and solution e Parallel structure 
e Comparison and contrast e Quotations 
Content features ¢ Examples 
e e —_- Point of view * Repenen 
3 e Topics or central ideas ¢ Logical arguments 
& e Supporting ideas and 
evidence 
Graphic features 
e Titles 
e  Subheadings 
e Italics 
e Captions 
= e Sidebars 
s e Photos and illustrations 
4 e Charts and tables 
As e = Historical document e —_ Irony 
e Essay (eg., informational, e Sarcasm 
a persuasive, analytical) 
% e Research report 
es) 
Plus increasingly complex Increasingly complex application | Plus increasingly complex 
application of grade 4 of grade 4 application of grade 4 
e Essay (eg., political, social, e Denotation 
historical, scientific, e  Connotation 
natural history) e Complex symbolism 
e — Literary analysis e Extended metaphor 
Q and analogy 
3 e ~~ Paradox 7 
& e Contradictions/ 
e —Incongruities 
e Ambiguity 
Plus increasingly complex applica- | Increasingly complex application | Increasingly complex application 
tion of grades 4 and 8 of grade 4 of grades 4 and 8 


Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 


features or techniques of author's craft. 
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Exposition 


As they progress beyond the early grades, students read expository text with increasing fre- 
quency both in and out of school (Broer, Aarnoutse, Kieviet, and Van Leeuwe 2002). The primary 
goals of expository text for school-age readers are to communicate information and to advance 
learning. Forms that may be assessed at grade 4 are informational trade books, textbook passages, 
news stories, feature stories, and encyclopedia entries. At grade 8, expository text genres include 
historical documents, various grade-appropriate essays, and research reports. More complex essay 
formats will be included for assessment at grade 12 such as political, social, historical, or scientific 


essays that primarily communicate information. 


Expository texts are characterized by internal sets of grammars similar in function to the story 
grammars discussed in chapter one. These grammars are designed to move the exposition forward 
and to help the reader comprehend the text. As shown in the matrix, the major organizational struc- 
tures of exposition are description, sequence, cause and effect, problem and solution, and compar- 
ison and contrast (Meyer 1975). As mentioned in chapter one, exposition may also include lists as 
a structural component with lists of descriptions, causes, problems, solutions, and views presented 
within other structures. Commonly, exposition does not contain just one structural format, but 


rather combines several structures embedded in the text. 


Specific elements within these organization structures signal meaning to the reader. Sequence, 
point of view, topics or central ideas, and supporting ideas and evidence are listed at grade 4; at 
grade 8 and grade 12, the structural organization and elements will be assessed at increasingly com- 
plex levels and with increasingly sophisticated texts. Some surface-level or graphic features support 
the text structures of exposition and guide the reader through the text. Other textual features can be 
categorized as reflecting author’s craft; these features guide the reader through the use of transitional 
words, signal words, voice, figurative language, and rhetorical structures. At grades 8 and 12, increas- 


ingly complex use of these features and of the author’s craft will be included for assessment. 
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Exhibit 4 (continued). Informational text matrix: Argumentation and persuasive text 


Genre/Type of Text Text Structures and Features Author’s Craft 
e Informational Organization e Transitional words 
trade book e Description e Signal words 
e Journal e Sequence (¢.g., enumeration, e Voice 
e = Speech chronology) e _ Figurative language and 
e Simple persuasive e Cause and effect rhetorical structure 
essay e Problem and solution e Parallel structure 
e Comparison and contrast e Quotations 
Content features e Examples 
e Author’s perspective or position e Repetition 
e Topics or central ideas e Exaggeration 
e Supporting ideas and evidence ¢ Emotional appeal 
= ; ; . e Tone 
“I e Contrasting viewpoints and 
3 
é perspectives 
e Presentation of the argument 
(eg., issue definition, issue 
EI choice, stance, relevance) 
= Graphic features 
ao e —_ Titles 
@ . 
= e Subheadings 
D 
o e Italics 
Se, 
2 e Captions 
= e Sidebars 
a e Photos and illustrations 
e 
= e Charts and tables 
oy) 
& e —_ Letter to the editor e Irony 
pp e Argumentative essay e = Sarcasm 
= M | Rima! d 
= ore complex per- e igurative language an 
a suasive essay rhetorical structure 
& e —- Editorial e Parallel structure 
e Quotations 
Plus increasingly complex Increasingly complex application of Plus increasingly complex 
application of grade 4 grade 4 application of grade 4 
e — Essay (e.g,, political, 
social) 
e ~—- Historical account 
N bee 
| e Position paper (e.g., 
3 persuasive brochure, 
rc) campaign literature, 
advertisements) 
Plus increasingly complex Increasingly complex application of Increasingly complex application 
application of grades 4 and 8 | grade 4 of grades 4 and 8 


Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 


features or techniques of author's craft. 
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Argumentation and Persuasive Text 


Many forms of informational text pose an argument or attempt to persuade readers toward a 
particular viewpoint. These texts present information to support or prove a point, to express an 
opinion, and to try to convince readers that a specific viewpoint is correct or justifiable. Various 
logical fallacies and forms of bias may be found in argumentation and persuasive text. As the matrix 
shows, there is considerable similarity in structure, literary features, and elements among exposition, 
argumentation, and persuasive text. The real distinction lies in the purpose for which an author 
writes these particular kinds of informational text; as stated, exposition seeks to inform and educate, 
whereas argumentation and persuasive texts seek to influence their readers’ thinking in other, often 


subtle but significant ways. 


At grade 4, argumentation and persuasive texts listed in the matrix are informational trade 
books that specifically argue a position or persuade the reader toward a stance, journals, speeches, 
and simple persuasive essays. However, in 2019, NAEP will not assess argumentation and persuasive 
texts at grade 4 due to difficulty in locating high-quality texts appropriate for this grade level. At 
grade 8, there are more complex forms of argumentation and persuasive texts: letters to the editor 
and editorials, and argumentative and grade-appropriate persuasive essays. At grade 12, argumen- 
tation and persuasive texts become increasingly complex with a variety of types of essays such as 
political and social commentary essays, historical accounts, and position papers such as persuasive 


brochures, campaign literature, and advertisements. 


Particular organization techniques and elements are used to create a clear argument or to form 
a persuasive stand. The differences between exposition and argumentation and persuasive text lie 
not in the structural organization, but rather in the way the texts are elaborated through the use of 
contrasting viewpoints, shaping of arguments, appeals to emotions, and other manipulations of the 
elements of text and language. The organizational structures at all levels are the same as in exposi- 
tion: description, sequence, cause and effect, problem and solution, and comparison and contrast; 


they are represented in grades 8 and 12 with increasing complexity. 


Elements within these organizational structures include the author’s perspective, topics or 
central ideas, supporting ideas, contrasting viewpoints or perspectives, and the presentation of the 
argument (e.g., issue definition, issue choice, stance, and relevance). These elements appear at all 
grade levels, with complexity increasing at higher grade levels. In addition, at grade 12, students 
may be asked about the structure of a given argument; connections among evidence, inferences, and 
claims; and the structure of a deductive versus an inductive argument. Twelfth-grade students may 
also be asked questions about the range and quality of evidence, logical fallacies, false assumptions 
and premises, loaded terms, caricature, leading questions, and faulty reasoning in argumentation and 


persuasive texts. 
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Exhibit 4 (continued). Informational text matrix: Procedural texts and documents 


Genre/Type of Text Text Structures and Text Features 
Embedded in text Organization 
e _ Directions e ~—- Description 
e Map e Procedures 
e Timeline e Sequence (e.g., enumeration, chronology) 
e «Graph Graphic features 
1.8) Table e —_ Titles 
3 e Chart e —_ Labels 
glo e Headings 
3 e  Subheadings 
5 e Sidebars 
ras e Photos and illustrations 
fe e Charts and graphs 
2 e —_ Legends 
i Embedded in text 
S o e = Recipe 
: g e Schedule 
£ Plus increasingly complex application of grade 4 Increasingly complex application of grade 4 
Stand-alone material 
e = Application 
a] Manual 
3 e Product support material 
fc e = Contract 
Plus increasingly complex application of grades Increasingly complex application of grades 4 and 8 
4and 8 


Note: Entries listed within each cell should be construed as neither definitive nor inclusive of all text structures and 


features or techniques of author's craft. 
Procedural Texts and Documents 


Research indicates that adults spend considerably more time reading documents (i.e., informa- 
tion in matrix or graphic form) than they do reading prose materials (Guthrie and Mosenthal 1987; 
Kirsch and Mosenthal 1990; Mosenthal 1996; Mosenthal 1998). Documents and procedural texts 
are indeed common in our society; for example, we interpret bus schedules, assemble simple devices, 
order goods from a catalog, or follow directions to set the VCR clock. Such texts are used frequently 
in elementary and secondary schools, where students encounter textbooks replete with graphs, 


tables, and illustrations to accompany and expand traditional continuous text. 


Procedural text may be primarily prose arranged to show specific steps toward accomplish- 


ing a goal, or it may combine both textual and graphic elements to communicate with the user. 
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Documents, in contrast, use text sparingly, in a telescopic way that minimizes the continuous prose 


that readers must process to gain the information they need. 


As the matrix shows, document texts on the NAEP Reading Assessment may include, but are 
not limited to, tables and charts. Stand-alone procedural text or documents will not be included at 
grades 4 and 8; such text will be embedded in or ancillary to continuous text. They may appear as 
stand-alone stimuli at grade 12 but their use will account for only a small amount of the stimuli in 
the entire assessment. It is likely that many of the documents may be used as part of intertextual item 
sets. For example, a student might encounter a bar graph and a timeline with items that relate to 
both texts. 


Documents and procedural text features act as necessary clues to the organization of the text. 
As textual supports, these features guide the reader through the text. For the purposes of the NAEP 
Reading Assessment, graphic features include titles, labels, headings, subheadings, sidebars, photos 
and illustrations, charts and graphs, and legends at grades 4, 8, and 12. More complex examples of 


these will be included at each successive grade. 


Characteristics of Texts Selected for Inclusion 


Passages selected as stimulus material for the NAEP Reading Assessment must meet rigorous 
criteria. They will all be authentic texts of the highest quality, evidencing characteristics of good 
writing, coherence, and appropriateness for each grade level. Passages will be drawn from a variety 
of contexts familiar to students nationwide. Stimulus material must be engaging to students at each 
grade level. Furthermore, material must reflect our literary heritage by including recognized works 
from varied historical periods (Ravitch 2003). 


It is true that children’s experience differs from that of adults, and therefore the appli- 
cation of standards should be consonant with child life. Nevertheless, one must keep 
in mind the emotional maturity of the children for whom the book or books are 
intended. This does not mean that the works must be watered down so as to meet 
the reading ability levels of young children. On the contrary, some books of lasting 
value outstrip their vocabulary lists and connect with children on emotional-maturity 
levels so that they can be understood and enjoyed by the young themselves. ... ['T ]he 


standards basic to good writing in adult literature are also basic to good writing for 


children (Georgiou 1988). 


Most material included on the assessment will be presented in its entirety as students would 


encounter it in their own reading. However, some material may be excerpted, for example, from 
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a novel or a long essay. Excerpted material will be carefully analyzed to ensure that it is coherent 


in structure. 


Passage Length 


Material on the assessment will be of differing lengths as shown in exhibit 5. 


Exhibit 5. Passage lengths for grades 4, 8, and 12 


Grade Range of Passage Lengths (Number of Words) 
4 200-800 
8 400-1,000 
12 500-1,500 


Passages of these lengths are used for several reasons. To gain the most valid information about 
students’ reading, stimulus material should be as similar as possible to what students actually 
encounter in their in-school and out-of-school reading. Unlike many common reading tests that 
use short passages, NAEP will present longer material that challenges students to use their strategic 
reading skills in ways that reflect the kinds of reading they do in nontest situations (Paris, Wasik, and 
Turner 1991). Furthermore, short passages usually will not yield approximately 10 distinct items, 
the required minimum number for each NAEP item set. Longer passages, with clear structural 
patterns, can support the development of multiple, distinct, nontrivial items that cover the range of 
content included in the literary and informational text matrices. These items will also allow broad 


coverage of the cognitive targets discussed later in this chapter. 


It is expected that in some cases, two poems will be used together to assess students’ ability to 
compare them in terms of their themes and stylistic features. Prose passages used in intertextual item 
sets will also be fairly short. Likewise, it is possible that two documents might be included as inter- 


textual stimuli at grade 12. 


Selection of Literary and Informational Passages 


Several methods of evaluating passages will be used to ensure that the best possible stimulus 
material is included. Authentic material must be of the highest quality, and it must come from 
authentic sources such as those students would encounter in their in-school and out-of-school read- 
ing. Material must be coherent and allow items that assess domain-specific knowledge (Kobayashi 
2002). Additionally, systematic efforts will be made to ensure that texts selected for inclusion will 
be of interest to the widest number of students. Readers become more engaged in text and conse- 


quently comprehend a selection better when they find the material interesting (Baumann 1986; 


31 | 


Wade, Buxton, and Kelly 1993; Wade and Moje 2000; Wade et al. 1993). Texts will reflect literary 


heritage by representing varied historical periods. 


Passages selected for inclusion on the assessment will be well written, interesting to read, and 
considerate; that is, easily comprehensible because they are well organized, have appropriate vocabu- 
lary, and, where needed, have useful supplemental explanatory features such as definitions of techni- 
cal terms or topographical features. Ideas marked by graphic features such as italics, bold print, and 
signal words and phrases tend to be processed more easily and recalled longer than unmarked infor- 
mation. In selecting passages, attention will be paid to written clues within text that can help readers 
understand structure, guide the development of main ideas, and influence the recall of information. 
For example, readers tend to organize and remember emphasized information better when authors 
lead them with signal words indicating main ideas (for example, the most important point here), with 
phrases indicating sequencing (such as words like first, second, third), and with statements cross-ref- 


erencing disparate parts of text (Armbruster 1984). 


Especially in the selection of informational text, the degree of content elaboration will be an 
important criterion for passage selection. Sufficient elaboration of new concepts is needed if stu- 
dents are to gain sufficient information to respond to questions. Tersely written informational text 
tends to be more difficult for students to comprehend compared with text written with more elabo- 
rated explanations. Whether text is tersely written or presents fully elaborated content is particularly 


important with topics that may be beyond the background knowledge of some students. 


An inviting writing style can also enhance interest and thereby increase comprehension. Mate- 
rial may be interesting not because of what is said but because of how it is said. For example, writers 
can increase interest by using active rather than passive verbs, by including examples that make the 
writing less abstract, and by using vivid and unusual words. An inviting writing style also influences 
voice. Voice, the qualities that help a reader view text as communication between an author anda 
reader, can have a positive effect on recall (Beck, McKeown, and Worthy 1995). 


Expert judgment will be the primary method for evaluating and selecting passages for inclusion 
on the assessment. Additional methods include the use of story and concept mapping and vocabu- 
lary mapping. At least two research-based readability formulas will also be used to gather additional 
information about passage difficulty (Klare 1984; White and Clement 2001). Passages will be thor- 


oughly reviewed for potential bias and sensitivity issues. 


Story and concept mapping procedures have been used to identify appropriate passages for 
previous assessments (Wixson and Peters 1987). These procedures result in a graphic representation 
of a possible stimulus selection that clearly highlights the hierarchical structure and the interrelat- 


edness of the passage components. Story mapping, for example, will show how the setting of a story 
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is related, and contributes to, the development of plot and theme. Concept mapping shows the 
structure of informational text along with the concepts presented and the relational links among 
concepts. Organizing information hierarchically within a passage allows the identification of various 
levels of information within a text so that items can target the most important aspects of what stu- 


dents read. 


As NAEP begins to assess vocabulary in a systematic way, the story and concept mapping proce- 


dures will be modified to ensure that appropriate words are selected for item development. 


Selection of Poetry 


In selecting poetry for the NAEP Reading Assessment, it will be important to determine that 
potential poems present a theme instead of stressing primarily the melodic or stylistic aspects of 
language use. Especially at grades 4 and 8, the theme should be implicitly presented in terms that are 
not so abstract that they are beyond students’ comprehension. Words and phrases should be used 
with economy to support and amplify the meaning inherent in the text; the style should be distin- 
guished by author’s craft and project the poet’s feelings about his or her topic or theme. The ideas 
presented must be accessible to students, and it must be clear that poetry, rather than prose, is the 


better mode for presenting these ideas. A good question to ask in selecting poetry is: 


Does the poetry, through its expression of theme and ideas, carry children beyond 
their immediate experiential level to extensions where language and imagination 


meet? (Georgiou 1988) 


Selection of Noncontinuous Text and Documents 


In addition to continuous text prose and poetry, the assessment will include prose augmented 
by noncontinuous textual elements such as embedded tables, charts, animations, or videos. It will 
also include stand-alone documents at grade 12. An analysis of layout will be essential to ensure that 
embedded noncontinuous text is used appropriately in a way that is well integrated into the prose 
text and not gratuitously distracting. Equally, stand-alone documents must be rich with appropriate 
information about which questions can be asked. The number of categories of information pre- 
sented graphically and the clarity of the layout of documents will be essential criteria for selecting 
documents for inclusion. The vocabulary and concept load of multimedia elements and of docu- 


ments will also be considered. 
Exhibit 6 summarizes the considerations for selecting passages and documents. The first two 


columns present considerations for literary and informational continuous text. The third column 


presents considerations that must be made in selecting noncontinuous text that is embedded within 
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continuous text or documents that will be used as stand-alone stimulus material at grade 12. Certain 
considerations are considered essential for each kind of stimulus material and represent the funda- 
mental characteristics that make a text or document appropriate for inclusion. All potential stimulus 
material must also be grade appropriate to ensure that students will be able to understand the con- 
cepts presented and are familiar with the material’s stylistic features. Finally, balance must be consid- 
ered so that the assessment as a whole reflects the full range of print and digital texts that students 


encounter in their in-school and out-of-school reading. 
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Exhibit 6. Considerations for selecting stimulus material 


Literary Text 


Essential characteristics 
e Ability to engage readers 
e = Well-written, rich text 
e Recognized literary merit 
e Theme/topic appropriateness 
by grade level 
Grade appropriateness 
e Complexity of characters 
e Number of characters 
e Vocabulary 
e Sophistication in use 
of literary devices 
e Complexity of dialogue 
e ~—- Point of view 
e Complexity of theme 
e = Multiple themes (major/minor) 
e Use of time (flashbacks, 
progressive/digressive) 
e Illustrations 


Balance 


e Reflective of our literary heritage 


e Style 

e Variety of sentence and 
vocabulary complexity 

e Appropriateness of mode 
(prose vs. poetry) 


e — Classical as well as contemporary 


e Representative of varied historical 


periods, cultures, socioeconomic 
backgrounds, etc. 


e Genre 


Informational Text 


Essential characteristics 
e Ability to engage readers 
e Well-written, considerate text 
e Coherence 
e Theme/ topic appropriateness 
by grade level 
Grade appropriateness 
e ‘Topic 
e Vocabulary 
e Concepts (number, 
familiarity, abstractness) 
e = Curricular appropriate- 
ness at grade level 
e Integrity of structure 
e Types of adjunct aids 
e _ Explicitness of perspective 
e Style 
Balance 
e Varied content areas 
e Style 
e Genre 
e Variety of sentence and 
vocabulary complexity 


e Appropriateness of mode 


Graphical Displays of 


Information 


Essential characteristics 
e Coherence 
e = Clarity 
e Relevance (when embedded) 
Grade appropriateness 
e Structural complexity 
e ‘Topic 
e Vocabulary 
e Concepts (number, familiar- 
ity, abstractness) 
e Number of categories of 
information presented 
e Amount of information 
within categories 
Balance 
e Embedded documents 
balanced with stand-alone 
documents (at grade 12) 


° Format 


Vocabulary on the NAEP Reading Assessment 


The 2019 NAEP Reading Assessment includes an assessment of vocabulary in the context 


of passages that students read. Vocabulary knowledge is considered to be one of the five essential 


components of reading as defined by the Every Student Succeeds Act (ESSA). In this context, 


vocabulary is construed not as isolated word meanings but as real knowledge of vocabulary that can 


advance comprehension. 
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NAEP will not test definitions in isolation from surrounding text; that is, students will not be 
assessed on their prior knowledge of definitions. The definition of meaning vocabulary will guide 
the development: 


Meaning vocabulary is the application of one’s understanding of word meanings to 
passage comprehension. 


Importance of Vocabulary for Reading Comprehension 


The associations between vocabulary and learning to read and then between vocabulary and 
reading comprehension are well documented in research (Hart and Risley 1995).° Studies have 
repeatedly shown that students’ vocabulary is a fundamental factor in their ability to comprehend 
what they read. Not knowing the meaning of words as used in a given text may result in decreased 
comprehension of that text. Comprehending any reading passage requires knowing the meaning of 
the important content-bearing words of that passage, but often, the meaning of many key words in 
a passage depends on an interaction of word meaning and passage meaning (Baumann, Kame’enui, 
and Ash 2002; Landauer, Foltz, and Laham 1998). Because of this interaction, measurement of 


word meaning by NAEP should be integrated with the measurement of passage comprehension. 


Several major factors are known to affect readers’ comprehension of what they read and can 


highlight the connection between word meaning and passage meaning; these include: 


e The context for reading (e.g., for study, for skimming, for leisure). 

e Fluency in identifying the words of the text. 

e Background or domain knowledge of the content of the text. 

e Knowledge of “the sense of the meaning” of the words the author uses to convey important 
content (Miller 1991). 


e Comprehension monitoring. 


Reasons for Assessing Vocabulary on NAEP Reading 


The growing body of research documenting the link between vocabulary and reading com- 
prehension provides a strong rationale for the inclusion of a systematic measure of vocabulary. 
Past assessments have included a few vocabulary test items, all of which measured vocabulary in 
context; however, the number of items was scant, and there were no specific vocabulary-related 
criteria for selecting the items or distractors. Furthermore, NAEP reports provided no information 


about performance on those items or how vocabulary performance might be related to reading 


> Fora complete list of references substantiating vocabulary assessment, see the bibliography. 
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comprehension. This change for the NAEP Reading Assessment, then, is significant. All vocabulary 
items will function both as a measure of passage comprehension and as a test of readers’ specific 


knowledge of the word’s meaning as intended by the passage author. 


Measurement of Meaning Vocabulary 


Vocabulary items will be developed about the meaning of words as they are used in the context 
of the passages that students read. Students will not be asked to draw on their prior knowledge by 
providing a written definition of each word on a list or in a set of words. There are two reasons for 
this approach. First, knowledge as explicit as a written definition of a word is not the specific ability 
required for passage comprehension. In reality, readers may not be able to provide a complete defi- 
nition of a word they encounter but do have enough sense of the word’s meaning as used in text that 


their comprehension is not impeded. 


A second argument against demanding specific definitions is that word meaning often depends 
on the context in which the word appears. Finding out whether readers know one specific definition 
of a word will not indicate whether they understand that word as it is used in a given text. Indeed, 
there is evidence that readers who know one definition of a word but not the meaning in a given text 
try to alter the sense of the text in keeping with their known definition; leading, of course, to mis- 
understanding the text (Deegan 1995). In addition, writers often use words in a manner that goes 
beyond concrete, familiar definitions but do so in ways that skilled readers can interpret effectively. 


Jacques Barzun describes this: 


Language is not an algebra; that is, the symbols do not stay put, nor can they be carried 
from place to place with an assurance that their value will not change. If language were 
like an algebra there could be no poetry or other fiction, no diplomacy or intimate 
correspondence, no persuasion or religious literature. If language were like an algebra, 
uncomfortable would mean not able to be comforted, and a myriad other nuances of 


human feelings would have to remain unrecorded and unshared (Barzun 1975). 


Considerations for Selecting Vocabulary 


In selecting passages, test developers must create a map of the story or expository selection to 
identify a passage’s key features. This procedure has included identifying candidates for vocabulary 
items, but the process will be enhanced to ensure that passages contain enough candidate words or 


terms for item development. 


The intent of the vocabulary assessment is to determine whether readers know and understand 


the meanings of the words that writers use to convey new information or meaning, not to measure 
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readers’ ability to learn new terms or words. Hence, the assessment will focus on words that charac- 
terize the vocabulary of mature language users and characterize written rather than oral language. 
The words selected for item development will convey concepts, ideas, actions, or feelings that the 
readers most likely know. In general, the words selected as targets for item development characterize 
the language of mature readers and are used in texts from a variety of content domains (Beck, McK- 
eown, and Kucan 2002).* Considerations for selecting words for item development are summarized 


in exhibit 7. 


Exhibit 7. Considerations for selecting vocabulary items and distractors 


Vocabulary Words 


; Considerations for Distractors 
Excluded From Testing 


Vocabulary Words to Be Tested 


Characterize the vocabulary 
of mature language users and 
characterize written rather 
than oral language. 

Label generally familiar and 
broadly understood concepts, 
even though the words them- 
selves may not be familiar to 
younger learners. 

Necessary for understanding 
at least a local part of the 
context and linked to cen- 
tral ideas such that lack of 
understanding may disrupt 
comprehension. 

Are found in grade-level 


reading material. 


Narrowly defined and not 
widely useful, such as those 
related to specific content 
domains (¢.g., photosyn- 
thesis, fiduciary) or words 
with limited application 

(eg., deserter, hamlet). 

Label or name the main idea 
of the passage (e.g., the word 
“emancipation” would not 

be tested in an article deal- 
ing with the “Emancipation 
Proclamation”). 

Already likely to be part of 
students’ everyday speaking 
vocabulary at grade level. 
Meanings readily derived from 
language context (e.g., appos- 
itives, parenthetic definitions, 


idiomatic expressions). 


Present a different common 
meaning of the target 
vocabulary word, which must 
be ignored in favor of the 
meaning in context. 

May present correct informa- 
tion or content from the text 
that is not what is meant by 
the target word. 

May be an alternative interpre- 
tation of the context in which 
the target word occurs. 

May be the meaning of another 
word that looks or sounds 
similar to the target word. 
May present a common but 
inaccurate association with the 


target word. 


Words that are appropriate for inclusion denote concepts or things that readers already know. 
That is, the word denotes an object, idea, feeling, or action that has been experienced or has been 
seen by the readers. However, the test item is not designed to determine whether readers know the 
thing, but rather whether readers are able to link this knowledge (object, idea, feeling, or action) 
to the word the author uses to convey this meaning. NAEP presumes that most readers are likely 
to have the background knowledge of the object, idea, feeling, or action in a passage, but because 
the words are difficult and uncommon, readers may not readily link that knowledge to the specific 


* Referred to as tier 2 words, a term that distinguishes them from tier 1 words, which are common, everyday words 
basic to the speech and writing of most students, and from tier 3 words, rarely used words or technical terminology. 
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word the author uses to convey that meaning. If readers do not connect a meaning with the author's 
word, their comprehension will suffer. NAEP vocabulary items are designed to test readers’ ability 
to connect an appropriate meaning to the candidate words to gain comprehension. Thus, test items 
will not target technical terms or words identifying the central idea(s) of the passage because those 
words often represent new knowledge, concepts, or conceptualizations for readers. Passage com- 
prehension items will measure readers’ learning from text; vocabulary items will measure readers’ 


knowledge of certain important words the author uses to impart this meaning. 


Clearly, some students will know and understand some test words before taking the assessment. 
This is unavoidable. Furthermore, we anticipate that some readers will not have the background to 
link to the author’s words and thus will either choose an incorrect response for the item because of 
their background knowledge or identify the meaning of the word from context and mark the correct 
response. Recognizing this possibility, NAEP will ensure that the vocabulary test items represent a 
continuum of difficulty across readers at a given grade (as will reading passages and comprehension 
items). The intent is to identify words that the majority of grade-level students do not generally use 


in speaking or writing but have seen or heard at least a few times. 


Cognitive Targets 


Items will be developed to assess students’ comprehension of literary and informational text. 
The term cognitive targets refers to the mental processes or kinds of thinking that underlie reading 
comprehension. Test questions will be aligned to cognitive dimensions applicable to literary and 
informational texts and also to cognitive dimensions specific to each text type. The remainder of the 
chapter presents those cognitive dimensions targeted by the items (hence the term cognitive tar- 
gets) and discusses the item types included on the assessment. Inclusion of specific cognitive targets 
reflects the intent of the definition of reading that guides the assessment. The definition, explained 


in chapter one, follows. 


Reading is an active and complex process that involves: 
e Understanding written text. 
e Developing and interpreting meaning. 


e Using meaning as appropriate to type of text, purpose, and situation. 


Reading Processes Included in Cognitive Target Matrices 


The reading processes included in the three sections of the cognitive target matrix, exhibit 8, 
illustrate the complex nature of reading. The research literature contains numerous studies that 
show how students use different reading processes when reading various types of text (see chapter 


one). Hence, the sections of the matrix representing literary and informational text emphasize that 
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different texts elicit different kinds of reading behaviors. The reading processes presented in the 
matrix are also grounded in the research literature on comprehension, most specifically the litera- 
ture that uses protocol analysis (think-alouds) as its research methodology (Garner 1982; Guthrie, 
Britten, and Barker 1991; Norris and Phillips 1987; Pressley and Afflerbach 1995; Olvshavsky 
1976-77). Furthermore, they reflect the cognitive processes assessed on international reading assess- 
ments such as the Progress in International Reading Literacy Study (PIRLS) (Campbell et al. 2001) 
and the Programme for International Student Assessment (PISA) (OECD 2000). The behaviors 
presented in exhibit 8 are illustrative, not comprehensive. The Specifications for the NAEP Reading 
Assessment will provide a detailed listing of the cognitive targets for item development. 


Locate and Recall 


The first cognitive behaviors are /ocate and recall. As students locate or recall information from 
what they read, they may identify clearly stated main ideas or supporting details, or they may find 
essential elements of a story, such as characters, time, or setting. Their process in answering assess- 
ment items often involves matching information given in the item to either literal or synonymous 
information in the text before they can then use the textual information to develop a response. As 
readers engage in these behaviors, they monitor their reading in order to understand when they are 
comprehending and when they are not. When they realize that the text is not making sense, they 


employ specific strategies to ensure that they begin to comprehend again. 


A salient activity [in reading] is to find the main ideas in the text and make certain that 
these ideas are remembered—or at least can be found again if needed. The big ideas, of 
course, are always relative to the goals of the reading with respect to the text. That is, 
very different ideas may be considered main ideas if a reader is reading for one purpose 


versus another (Pressley and Afflerbach 1995, p. 44). 


Items assessing this component of reading usually focus on information contained in relatively 
small amounts of text: a sentence, a paragraph, or two or more adjacent paragraphs. These items 
provide information about the most basic comprehension skills, those that ultimately form the 
foundation for a more elaborate understanding of what is read. At the same time, these items address 


the kinds of reading that occur routinely in school and in out-of-school reading activities. 


Regardless of a reader’s goal—whether reading is done in preparation for a test, in 
anticipation ofa writing assignment, with the expectation of sharing it in a conversation, 
to determine an author's perspective, or as part of staying abreast in an area of interest— 
it is necessary to identify the important information in a text (Pressley and Afflerbach 


1995.93): 
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Integrate and Interpret 


The next set of reading behaviors refers to what readers do as they integrate new information 
into their initial sense of what a passage says, often interpreting what they read in the process. When 
readers engage in behaviors involving integrating and interpreting, they make comparisons and con- 
trasts of information or character actions, examine relations across aspects of text, or consider alter- 
natives to what is presented in text. This aspect of the reading is critical to comprehension and can 
be considered the stage at which readers really move beyond the discrete information, ideas, details, 
themes, and so forth presented in text and extend their initial impressions by processing informa- 
tion logically and completely. As readers integrate information and interpret what they read, they 
frequently form questions, use mental images, and make connections that draw on larger sections of 
text, often at an abstract level. They also draw on their knowledge of the structure and elements of 


literary and informational text. 


In applying these behaviors, readers invariably think across large portions of text, across the text 
as a whole, or even across multiple texts; they relate textual information to knowledge from other 
sources such as their previous content learning or to internalized criteria and logic. Thus, readers 
might ask themselves whether something they are reading makes sense to them within the realm of 
their own experiences or when considered against what they have read in other sources. They exam- 
ine the text in terms of their specific reading goals or the needs they have for the information that 
the text can provide. In certain reading situations, readers may apply what they know to what they 
are reading, for example, determining a real-world application of suggestions in a text on bicycle 
safety. They also apply information gained from reading, for example, in following instructions for 


repairing a bicycle or reading a map to determine where bike routes have been designated in a city. 


Readers are aware of many different aspects of text and the reading task they are per- 
forming from the outset of reading. Their perceptions of the text and how it relates to 
their task/reading goals does much to shape the processing of text, with readers pro- 
cessing some parts of the text superficially and others very carefully. ... Good readers 
not only know what they are doing but also why they are doing it, ever aware of the 


characteristics of text they are confronting and their own reading goals (Pressley and 


Afflerbach 1995, p. 68). 


Items assessing these behaviors might ask students to form generalizations about a piece of 
informational text or make statements about how the setting of a story contributes to the creation 
of theme. Other items might require interpretation, for example, of a character’s motivations or of 
an author’s reasons for attempting to persuade readers about an issue. Other questions might ask for 
alternative actions that a character might have taken or an interpretation of an implied message or 


moral froma story. 
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Critique and Evaluate 


The final set of reading behaviors, critiquing and evaluating text, requires readers to stand back 
from what they read and view the text objectively. The focus remains on the text itself but the 
reader’s purpose is to consider the text critically by assessing it from numerous perspectives and 
synthesizing what is read with other texts and other experiences. Items may ask students to evaluate 
the quality of the text as a whole, to determine what is most significant in a passage, or to judge the 
effectiveness of specific textual features to accomplish the purpose of the text (e.g., the effectiveness 
of details selected to support a persuasive argument). Items might ask for the likelihood that an 
event could actually have taken place, the plausibility of an argument, or the adequacy of an expla- 
nation for an event. Items can ask students to focus at the level of language choices (for example, 
nuances expressed in a metaphor) or at the broader level of the entire text (for example, evaluating 


the effectiveness of an author’s craft to accomplish his or her overall goals). 


To answer these questions, students draw on what they know about text, language, and the ways 


authors manipulate language and ideas to achieve their goals. 


Sometimes readers recognize from the very start that they are likely to be evaluative 
with respect to a text, and likely to react to it affectively. ... Although some readers evi- 
dence great consistency in their evaluative stances as they read some texts, evaluations 
are often much more discriminated. Regardless of whether a reader is globally positive, 


globally negative, or a mixture of both, evaluations occur with respect to the style and 


context of text (Pressley and Afflerbach 1995, p. 76). 


Assessing Cognitive Targets 


Exhibit 8 presents the cognitive target matrix for the development of items to be used on the 
NAEP Reading Assessment.’ The term cognitive targets is used to refer to the mental processes or 
kinds of thinking that underlie reading comprehension; the cognitive targets serve to guide the 
test development process in that item writers target these processes or kinds of thinking as they 
write items. The cognitive targets remain the same across all three grades on the assessment, but the 
passages and documents about which items are developed will be of increasing sophistication at 


each grade. 


> The cognitive targets matrix is for illustrative purposes only and should not be considered an exhaustive list. The 
cognitive targets will be elaborated further in the Specifications for the NAEP Reading Assessment. 
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Exhibit 8. Cognitive targets 


Locate/Recall 


Integrate/Interpret 


Critique/Evaluate 


Both Literary and 
Informational Text 


Identify textually explicit 
information and make simple 
inferences within and across 
texts, such as: 

e ~—_ Definitions 

e —- Facts 


e Supporting details 


Make complex inferences within 


and across texts to: 


Describe problem and solu- 
tion or cause and effect 
Compare or connect ideas, 
problems, or situations 
Determine unstated assump- 
tions in an argument 
Describe how an author 
uses literary devices 


and text features 


Consider text(s) critically to: 


Judge author’s craft 

and technique 

Evaluate the author’s 
perspective or point of view 
within or across texts 

Take different perspectives 


in relation to a text 


Specific to Literary Text 


Identify textually explicit 
information within and across 
texts, such as: 

e Character traits. 

e Sequence of events 

or actions 
e —- Setting 
e Identify figurative 


language 


Make complex inferences within 


and across texts to: 


Infer mood or tone 
Integrate ideas to 

determine theme 

Identify or interpret a 
character’s motivations 

and decisions 

Examine relations 

between theme and 

setting or characters 

Explain how rhythm, rhyme, 
or form in poetry contribute 


to meaning 


Consider text(s) critically to: 


Evaluate the role of 
literary devices in 
conveying meaning 
Determine the degree 

to which literary devices 
enhance a literary work 
Evaluate a character’s 
motivations and decisions 
Analyze the point of view 
used by the author 


Specific to Informational Text 


Identify textually explicit 
information within and across 
texts, such as: 

e Topic sentence 

or main idea 

e Author’s purpose 

e = Causal relations 

e Locate specific informa- 


tion in text or graphics 


Make complex inferences within 


and across texts to: 


Summarize major ideas 
Draw conclusions and pro- 
vide supporting information 
Find evidence in support of 
an argument 

Distinguish facts 

from opinions 

Determine the importance 
of information within and 


across texts 


Consider text(s) critically to: 


Analyze the presentation 
of information 

Evaluate the way the 
author selects language 

to influence readers 
Evaluate the strength and 
quality of evidence used by 
the author to support his or 
her position 

Determine the quality of 
counterarguments within 
and across texts 

Judge the coherence, logic, 


or credibility of an argument 
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Items will be developed to assess all cognitive targets at each grade level but the distribution of 
cognitive targets will vary across grades. In determining the distribution across grade levels, careful 
thought was given to the kinds of texts that students encounter at each level. Reference was also 
made to the distribution across reading processes in the two international reading assessments, PISA 
and PIRLS (Campbell et al. 2001; OECD 2000). Exhibit 9 displays the distribution of cognitive 
targets across grades 4, 8, and 12. 


Exhibit 9. Percentage distribution of cognitive targets by grade 


Grade Lecate/ Recall Integrate/ Critique/ 
Interpret Evaluate 
4 30 50 0 


Item Types 


The NAEP Reading Assessment will include both selected-response and constructed-response 
items, and beginning in 2017, these item types will be presented in a digital platform. The transition 
to digital administration provides opportunities to expand the range of formats used for these types 
of items. These item types yield valuable information about students’ reading and allow a rich, full 
description of how the nation’s students approach different kinds of text. The inclusion of multiple 
types of items affirms the complex nature of the reading process because it recognizes that different 
kinds of information can be gained from each item type. It also acknowledges the real-world skill of 


being able to write about what one has read. 


The transition to digital administration also provides opportunities for the use of texts from 
digital sources, though items must minimize the extent to which navigation skills are required to 
respond correctly. When digital texts include dynamic features such as videos and hyperlinks, items 


related to the dynamic features in texts must be linked to the understanding of the written material. 


Selected-response items have a variety of formats, some of which allow for more than one 
correct response. Although the different formats require different amounts of time to complete, it 
is expected that these items will take most students approximately one minute to complete. Short 
constructed-response items can be answered by one or two phrases or by one or two sentences; they 
should take students approximately two to three minutes to complete. Extended constructed-re- 
sponse items should elicit longer, more elaborated answers of a paragraph or two. They should take 
students approximately five minutes to complete. Scoring rubrics for short and extended construct- 
ed-response items will focus on the content included in answers, not on spelling or grammatical con- 


siderations. However, students must answer constructed-response questions by using information 
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from the text to receive credit. Details regarding the scoring and short and extended constructed-re- 
sponse items appear in the Specifications for the NAEP Reading Assessment.® 


The distribution of selected-response and constructed-response items will vary across the grades 
assessed by the NAEP Reading Assessment. The percentages in exhibit 10 refer to the amount of 
assessment time that students will spend responding to these particular kinds of items. Hence, grade 
4 students will spend approximately 50 percent of the assessment time responding to selected-re- 
sponse items and 50 percent of the assessment time preparing written responses. Students at grades 8 
and 12 will spend more time preparing written responses. 


Approximately two items per passage will assess vocabulary knowledge. These items may be 
either selected response or short constructed response in format. Exhibit 10 shows the distribution 


of time to be spent on each kind of item. 


Exhibit 10. Percentage distribution of time to be spent on specific item types 


Short Constructed Extended Constructed 
Grade Selected Response 
Response Response 
4 50 40 10 
8 40 45 15 
12 40 45 15 


Less time is allocated to constructed-response items at grade 4 to reflect developmental differ- 


ences. Students at grade 4 may not be as familiar with written responses to reading questions as are 
older students (Kobayashi 2002). 


° The specifications will guide the development of the assessment. It will provide detailed information about the kinds 
of reading selections to be included, item types, and scoring criteria for constructed-response items. The specifications 


will also discuss test administration procedures, any considerations to be given to special populations, and special studies 
to be conducted in conjunction with the assessment (see appendix C). 
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CHAPTER THREE 


REPORTING RESULTS 


Results of the NAEP Reading Assessment administrations are reported in terms of average 
scores for groups of students on the NAEP 0-500 scale and as percentages of students who attain 
each of the three achievement levels (Basic, Proficient, and Advanced) discussed below. This is an 
assessment of overall achievement, not a tool for diagnosing the needs of individuals or groups 
of students. Reported scores are always at the aggregate level; by law, scores are not produced for 
individual schools or students. Results are reported for the nation as a whole, for regions of the 
nation, for states, and for large districts that volunteer to participate in the NAEP trial urban district 
assessment (TUDA). 


Legislative Provisions for NAEP Reporting 


Under the provisions of the Every Student Succeeds Act (ESSA) legislation, states receiving 
Title I grants must include assurance in their state plans that they will participate in the reading and 
mathematics state NAEP at grades 4 and 8. Local districts that receive Title I funds must agree to 
participate in biennial NAEP reading and mathematics administrations at grades 4 and 8 if they 
are selected to do so. Their results will be included in state and national reporting. Participation in 
NAEP will not substitute for the mandated state-level assessments in reading and mathematics at 


grades 3 to 8. 


In 2002, NAEP initiated TUDA in five large urban school districts that are members of the 
Council of the Great City Schools (the Atlanta City, City of Chicago, Houston Independent, Los 
Angeles Unified, and New York City Public Schools districts). Ten large districts participated in 
2003 and 2005. The number of districts participating in TUDA has grown over time to a total of 27 
in 2019. With student performance results by district, participating TUDA districts can use results 


for evaluating their achievement trends and for comparative purposes. 


Through ESSA and the NAEP TUDA program, the 2019 NAEP Reading results will report 
student achievement for the nation, states, and select large urban districts, enabling comparisons 


between states, large urban districts, and various student demographic groups. 
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Achievement Levels 


Since 1990, the National Assessment Governing Board has used student achievement levels for 
reporting results on NAEP assessments. The achievement levels represent an informed judgment of 
“how good is good enough’ in the various subjects assessed. Generic policy definitions for achieve- 
ment at the Basic, Proficient, and Advanced levels describe in very general terms what students at each 
grade level should know and be able to do on the assessment. Reading achievement levels specific to 
the NAEP Reading Framework were developed to elaborate on the generic definitions. New read- 
ing-specific achievement level descriptors replaced those aligned to the previous framework (Gov- 
erning Board 2003). Preliminary achievement level descriptors were developed for the assessment 
as a whole and for the vocabulary component of the assessment. These preliminary achievement 
levels were used to guide item development and initial stages of standard setting. The preliminary 


achievement level descriptions were refined as a result of the achievement level setting process. 


Exhibit 11 presents the generic achievement level descriptors. See appendix B for the final 
achievement level descriptions. 


Exhibit 11. Generic NAEP achievement levels 


Achievement Level Policy Definition 
NAEP Advanced This level signifies superior performance beyond NAEP Proficient. 
NAEP Proficient This level represents solid academic performance for each grade assessed. Students 


reaching this level have demonstrated competency over challenging subject 
matter, including subject matter knowledge, application of such knowledge to 
real-world situations, and analytical skills appropriate to the subject matter. 


NAEP Basic This level denotes partial mastery of prerequisite knowledge and skills 


that are fundamental for performance at the NAEP Proficient level. 


Reporting NAEP Results 


NAEP Reading Assessment results are reported in terms of average scores for groups of students 
on the NAEP 0-500 scale and as percentages of students who attain each of the three achievement 
levels (Basic, Proficient, and Advanced). Information is also provided about students who score below 
Basic. These students are not necessarily nonreaders; many can complete some tasks on the assess- 


ment but are not able to attain the minimum score required for Basic. 


Data are reported on subgroups of students by gender, race/ethnicity, eligibility for free or 
reduced-price lunch, region of the country, type of community, public or nonpublic school, and 
other variables of interest. Data are never provided for individual students or schools. Subscores 
should be provided for literary and informational texts. Results will also be provided about students’ 


responses to the vocabulary items. 
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The 2019 results will continue to use a 0-500 cross-grade scale. Use of such a scale affirms that 
reading is a development process and that students’ reading skills mature throughout their school 


years as they read increasingly diverse and sophisticated texts. 


The primary vehicle for reporting NAEP reading results is the NAEP Reading Report Card 
issued after each assessment administration. The report provides detailed information on the 
assessments, the students who participated, and the assessment results. Results are disaggregated 
by specific groups and are also presented for states that participate in the NAEP state assessment. 
Among the focal groups are males and females, students from various racial/ethnic backgrounds, 


and students who took the assessment with and without accommodations. 


NAEP data and information about the assessments are also available electronically through the 
National Assessment Governing Board (www.nagb.org) and the National Center for Education Sta- 
tistics/ NAEP (nces.ed.gov) websites. Furthermore, the NAEP Data Explorer can be used by inter- 
ested education administrators, researchers, and other stakeholders to develop focused reports. The 
NAEP e-Library (nces.ed.gov) provides access to other information such as NAEP reports, sample 
assessment passages, items, scoring rubrics with student-constructed responses, and data sources for 


more in-depth analysis of student achievement results or of the assessments themselves. 


Reporting State NAEP Results 


As discussed above, states receiving Title I funding must participate in the NAEP Reading 
Assessment at grades 4 and 8. Results are reported in the aggregate for participating students and 
are also disaggregated for specific reference groups of students. Individual state reports are generated 
in addition to reports that contrast results from participating states and from the nation as a whole. 
The NAEP Report Generator allows state and local administrators and others to customize reports 


and to investigate specific aspects of student reading achievement. 


Reporting Trend Data 


According to NAEP law and Governing Board policy, long-term trend assessments are con- 
ducted as part of NAEP in order to continue the national trend reports. In reading, long-term 
assessments have been administered since 1971. The long-term trend reports provide the only con- 
tinual measures of student achievement over such extended periods of time. Passages and accompa- 
nying test items administered as part of the long-term trend assessments have remained unchanged 


from their initial administration in 1971. 
The 2009-2019 NAEP Reading Framework represents several important changes from the 


framework that guided the assessment from 1992 to 2007 (see exhibit 2). The 2019 NAEP Read- 
ing Assessment will use the same framework used in 2009. The Reading Framework for the 2009 
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National Assessment of Educational Progress replaced the framework first used for the 1992 reading 
assessment and then for subsequent reading assessments through 2007. Compared with the previ- 
ous framework, the 2009 reading framework included more emphasis on literary and informational 
texts, a redefinition of reading cognitive processes, a new systematic assessment of vocabulary knowl- 


edge, and the addition of poetry to grade 4. 


The 2009 NAEP Reading Report Card included trend data on student reading performance 
from 1992 to 2009. Results from special analyses determined that the 2009 reading assessment 
results could be compared with those from earlier assessment years. These special analyses started 
in 2007 and included in-depth comparisons of the frameworks and the test questions, as well as an 
examination of how the same students performed on the 2009 assessment and the earlier assessment. 
A summary of these special analyses and an overview of the differences between the previous frame- 
work and the 2009 framework are available on the Web at nces.ed.gov/nationsreportcard/reading/ 
trend_study.asp. The 2019 NAEP Reading Report Card will continue to report trends in student 


reading performance extending from 1992. 


NAEP reports are useful in providing trend results over time to inform decisions and allocations 


of resources and framing of policy about reading. The questions that NAEP addresses include these: 


e Are students improving in reading achievement over time? 
e Are percentages of students at the upper achievement levels increasing, decreasing, or 
remaining the same? 


e Are the gaps in achievement among various groups narrowing? 


Assessments aligned to the 1992 framework and its subsequent versions have yielded trend data 
from seven national and six state administrations as shown in exhibit 12. 
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Exhibit 12. NAEP reading assessment administrations for 1992-2007 framework 


Year Grades for National Administration Grades for State Administration | 
1992 4,8, 12 4 | 
1994 4,8, 12 4 | 
1998 4,8, 12 4,8 | 
2000 4 

2003 4, 8,12 4,8 

2005 4,8, 12 4,8 | 
2007 4,8 4,8 | 

Contextual Variables 


Students participating in the NAEP assessments respond to survey questionnaires that gather 
information on variables important to understanding reading achievement nationwide. Teachers 
and school administrators also complete questionnaires. To the extent possible, information is also 
gathered from non-NAEP sources such as state, district, or school records to minimize the burden 
on those asked to complete the questionnaires. Questions are nonintrusive; free from bias; secular, 


neutral, and nonideological; and do not elicit personal feelings, values, or attitudes. 


As stated in Governing Board policy, the collection of contextual data on students, teachers, 
and schools is necessary to fulfill the statutory requirement that NAEP include information when- 
ever feasible that is disaggregated by race or ethnicity, socioeconomic status, gender, disability, and 
limited English proficiency. Contextual information serves the additional purpose of enriching the 
reporting of NAEP results by examining factors related to academic achievement in the specific 


subjects assessed. 


To satisfy the goal of enriching reports on student achievement in reading, contextual variables 
are selected to be of topical interest, timely, and directly related to academic achievement. The 
selection of variables about which questions will be developed may reflect current trends in the field, 
such as the use of technology in reading instruction or the extent to which students use the Internet 
as a reference tool. Recommendations on contextual variables for the NAEP Reading Assessment 


were presented as a separate document. 
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APPENDIX A 


GLOSSARY OF TERMS 


This glossary provides brief definitions of terms used throughout the NAEP Reading Frame- 
work. The terms are defined according to their use in the framework. The list includes terms that 
relate to types of reading materials, text structures and features, techniques of author’s craft, and 
other key terms. 

Allegory: Story in which the characters, settings, and events stand for abstract moral concepts. 
Alliteration: Repetition of initial consonant sounds. 

Allusion: Reference to a mythological, literary, or historical person, place, or thing. 

Analogy: Comparison of two things to show their likenesses in certain respects. 


Argumentation: Writing that seeks to influence through appeals that direct readers to specific goals 


or try to win them to specific beliefs. 

Audience: Writer’s targeted reader or readers. 

Author’s craft: Specific techniques that an author chooses to relay an intended message. 
Autobiography: Written account of the author’s own life. 

Ballad: Song or songlike poem that tells a story. 

Biography: Account of a person’s life written by another person. 


Causation: Text structure that presents causal or cause and effect relationships between the ideas 


presented in the text. 


Cognitive target: Mental process or kind of thinking that underlies reading comprehension; cumu- 


latively, the cognitive targets will guide the development of items for the assessment. 
Coherence: Continuity of meaning that enables others to make sense of a text. 
Comic relief: Event or character that serves as an antidote to the seriousness of dramatic events 


Comparison: Text structure in which ideas are related to one another on the basis of similarities 
and differences. The text presents ideas organized to compare, to contrast, or to provide an alterna- 


tive perspective. 
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Conflict: Struggle or clash between opposing characters, forces, or emotions. 


Connotation: Implicit rather than explicit meaning of a word. It consists of the suggestions, associ- 


ations, and emotional overtones attached to a word. 


Description: Text structure that presents a topic, along with the attributes, specifics, or setting 


information that describe that topic. 
Denotation: Exact, literal definition of a word independent of any emotional association or second- 
ary meaning. 


Detail: Fact revealed by the author or speaker that supports the attitude or tone in a piece of poetry 


or prose. In informational text, details provide information to support the author's main point. 
Diction: Word choice intended to convey a certain effect. 
Elegy: Poem that mourns the death of a person or laments something lost. 


Epic: Long narrative poem that relates the great deeds of a hero who embodies the values of a par- 


ticular society. 


Exaggeration or hyperbole: Deliberate, extravagant, and often outrageous overstatement. It may be 


used for either serious or comic effect. 


Exposition: One of the classifications of discourse whose function is to inform or to instruct or to 
present ideas and general truths objectively. Exposition presents information, provides explanations 


and definitions, and compares and contrasts. 
Fable: Brief story that teaches a moral or practical lesson about life. 


Fantasy: Story employing imaginary characters living in fictional settings where the rules of the real 


world are altered for effect. 


Fiction: Imaginative literary works representing invented rather than actual persons, places, 


and events. 


Figure of speech: Word or phrase that describes one thing in terms of something else, often involv- 


ing an imaginative comparison between seemingly unlike things. 

Flashback: Scene that interrupts the action of a work to show a previous event. 
Fluency: Ability to read text quickly and accurately and comprehend what is read. 
Foil: Character who sets off another character by strong contrast. 


Folktale: Short story from the oral tradition that reflects the mores and beliefs of a 


particular culture. 
Foreshadowing: Use of hints or clues in a narrative to suggest future action. 
Free verse: Poetry that has no regular meter or rhyme scheme. 


Genre: Category used to classify literary and other works by form, technique, or content. 
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Grammar: Coherent text structure on which readers rely as they seek to understand what they read; 


often referred to as story grammar. 


Graphic: Pictorial representation of data or ideas using columns, matrices, or other formats. Graph- 
ics can be simple or complex, present information in a straightforward way as in a list or pie graph, 
or embed or nest information within the document's structure. Graphics may be included in texts or 


be stand-alone documents (grade 12 only). 


Historical fiction: Story that re-creates a period or event in history and often uses historical figures 


as characters. 


Iambic pentameter: Line of poetry made up of five metrical feet or units of measure, consisting of 


an unstressed syllable followed by a stressed syllable. 


Imagery: Multiple words or a continuous phrase that a writer uses to represent persons, objects, 


actions, feelings, and ideas descriptively by appealing to the senses. 


Inference: Act or process of deriving logical conclusions from premises known or assumed to be 


true; the conclusions drawn from this process. 


Irony: Tension that arises from the discrepancy, either between what one says and what one means 
(verbal irony), between what a character believes and what a reader knows (dramatic irony), or 


between what occurs and what one expects to occur (situational irony). 
Legend: Inscription or title on an object (e.g., a key to symbols used on a map). 
Literary device: Literary technique used to achieve a particular effect. 


Literary heritage: Works by authors whose writing influenced and continues to influence the pub- 
lic language, thinking, history, literary culture, and politics of this nation. These works comprise the 


literary and intellectual capital drawn on by later writers. 


Literary nonfiction: Text that conveys factual information. The text may or may not employ a nar- 


rative structure and characteristics such as dialogue. 
Lyrical poetry: Poems that focus on expressing emotions or thoughts. 


Meaning vocabulary: Application of one’s understanding of word meanings to passage 


comprehension. 
Memoir: Type of autobiography that usually focuses on a single time period or historical event. 
Metaphor: Comparison of two unlike things without the use of “like” or “as.” 


Mixed text: Text that employs literary techniques usually associated with narrative or poetry while 
also presenting information or factual material, with the dual purpose of informing and offering 


reading satisfaction; requires readers to discern bias from fact. 
Monologue: Long, formal speech made by a character. 


Mood: Atmosphere or predominant emotion in a literary work. 
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Motivation: Circumstance or set of circumstances that prompts a character to act a certain way or 


that determines the outcome of a situation or work. 

Myth: Traditional story accepted as history, which serves to explain the worldview of a people. 
Narration: Telling of a story in writing. 

Narrative poetry: Poems that tell a story in verse, often focusing on a single incident. 

Ode: Long lyric poem on a serious subject often for ceremonial or public occasions. 
Onomatopoeia: Use of words that mimic the sounds they describe; imitative harmony. 
Parody: Imitation of a work of literature, art, or music for amusement or instruction. 


Parallel structure: Repetition of words, phrases, or sentences that have the same grammatical struc- 


ture or that restate a similar idea. 
Personification: Metaphor that gives inanimate objects or abstract ideas human characteristics. 
Perspective: Position, stance, or viewpoint from which something is considered or evaluated. 


Persuasion: Form of discourse whose function is to convince an audience or to prove or refute a 


point of view or an issue. 
Plot: Sequence of events or actions in a short story, novel, or narrative poem. 


Point of view: Perspective or vantage point from which a literary work is told or the way in which 


the author reveals characters, actions, and ideas. 


Problem/solution: Text structure in which the main ideas are organized into two parts: a problem 
and a subsequent solution that responds to the problem or a question and an answer that responds 


to the question. 


Procedural text: Text that conveys information in the form of directions for accomplishing a task. 
A distinguishing characteristic of such text is that it is composed of discrete steps to be performed in 


a strict sequence with an implicit end product or goal. 


Protagonist: Central character of a short story, novel, or narrative poem. The antagonist is the char- 


acter who stands directly opposed to the protagonist. 


Purpose: Specific reason or reasons for the writing. It conveys what the readers have to gain by read- 


ing the selection. Purpose is the objective or the goal that the writer wishes to establish. 


Repetition: Deliberate use of any element of language more than once: sound, word, phrase, sen- 


tence, grammatical pattern, or rhythmical pattern. 
Rhetoric: Art of using words to persuade in writing or speaking. 


Rhetorical device: Technique used by writers to persuade an audience. 
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Rhyme: Repetition of sounds in two or more words or phrases that appear close to each other in a 
y) 
poem. End rhyme occurs at the end of lines; internal rhyme, within a line. Slant rhyme is approxi- 


mate rhyme. A rhyme scheme is the pattern of end rhymes. 

Rhythm: Regular recurrence and speed of sound and stresses in a poem or work of prose. 
Sarcasm: Use of verbal irony in which a person appears to be praising something but is actually 
insulting it. 

Satire: Prose in which witty language is used to convey insults or scorn. 

Sequence: Text structure in which ideas are grouped on the basis of order or time. 

Setting: Time and place in which events in a short story, novel, or narrative poem take place. 
Simile: Comparison of two different things or ideas through the use of the words “like” or “as.” 
Sonnet: Fourteen-line lyric poem, usually written in iambic pentameter. 

Stanza: Division of a poem, composed of two or more lines. 

Style: Writer’s characteristic manner of employing language. 


Symbol: Object, person, place, or action that has both a meaning in itself and that stands for some- 


thing larger than itself, such as a quality, attitude, belief, or value. 
Syntax: Arrangement of words and order of grammatical elements in a sentence. 
Tall tale: Improbable, incredible, or fanciful story. 


Theme: Central meaning of a literary work. A literary work can have more than one theme. Most 


themes are not directly stated but rather are implied. A literary theme is not the same as a topic. 


Tone: Writer's or speaker’s attitude toward a subject, character, or audience conveyed through the 


author's choice of words and detail. Tone can be serious, humorous, sarcastic, objective, etc. 
Trait: Distinguishing feature, as of a person’s character. 


Understatement: Kind of irony that deliberately represents something as being much less than it 


really is; the opposite of hyperbole or overstatement. 


Voice: Distinctive style or manner of expression of an author or of a character. 
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APPENDIX B 


NAEP READING ACHIEVEMENT LEVEL DEFINITIONS 


The National Assessment of Educational Progress (NAEP) reading achievement level descrip- 
tions present expectations of student performance in relation to a range of text types and text 
difficulty and in response to a variety of assessment questions intended to elicit different cognitive 
processes and reading behaviors. The specific processes and reading behaviors mentioned in the 
achievement level descriptions are illustrative of those judged as central to students’ successful 
comprehension of texts. These processes and reading behaviors involve different and increasing 
cognitive demands from one grade and performance level to the next as they are applied within 
more challenging contexts and with more complex information. Although similar reading behaviors 
are included at the different performance levels and grades, it should be understood that these skills 
are being described in relation to texts and assessment questions of varying difficulty. Bold text is a 


short, general summary to describe performance at each achievement level. 


NAEP Reading Achievement Levels—Grade 4 
NAEP Fourth-grade students performing at the Basic level should be able to locate 


Basic relevant information, make simple inferences, and use their understanding of the 
(208) text to identify details that support a given interpretation or conclusion. Students 


should be able to interpret the meaning of a word as it is used in the text. 


When reading literary texts such as fiction, poetry, and literary nonfiction, fourth- 
grade students performing at the Basic level should be able to make simple inferences 
about characters, events, plot, and setting. They should be able to identify a problem 


in a story and relevant information that supports an interpretation of a text. 


When reading informational texts such as articles and excerpts from books, fourth- 

g 
grade students performing at the Basic level should be able to identify the main 
purpose and an explicitly stated main idea, as well as gather information from various 


parts of a text to provide supporting information. 
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NAEP Fourth-grade students performing at the Proficient level should be able to inte- 
Proficient grate and interpret texts and apply their understanding of the text to draw con- 


(238) clusions and make evaluations. 


When reading literary texts such as fiction, poetry, and literary nonfiction, fourth- 
grade students performing at the Proficient level should be able to identify implicit 
main ideas and recognize relevant information that supports them. Students should 
be able to judge elements of author’s craft and provide some support for their judg- 


ment. They should be able to analyze character roles, actions, feelings, and motives. 


When reading informational texts such as articles and excerpts from books, fourth- 
grade students performing at the Proficient level should be able to locate relevant 
information, integrate information across texts, and evaluate the way an author pres- 
ents information. Student performance at this level should demonstrate an under- 
standing of the purpose for text features and an ability to integrate information from 
headings, text boxes, graphics and their captions. They should be able to explain a 


simple cause-and-effect relationship and draw conclusions. 


NAEP Fourth-grade students performing at the Advanced level should be able to make 

Advanced complex inferences and construct and support their inferential understanding of 

(268) the text. Students should be able to apply their understanding of a text to make 
and support a judgment. 


When reading literary texts such as fiction, poetry, and literary nonfiction, fourth- 
grade students performing at the Advanced level should be able to identify the theme 
in stories and poems and make complex inferences about characters’ traits, feelings, 
motivations, and actions. They should be able to recognize characters’ perspectives 
and evaluate character motivation. Students should be able to interpret characteristics 


of poems and evaluate aspects of text organization. 


When reading informational texts such as articles and excerpts from books, fourth- 
grade students performing at the Advanced level should be able to make complex 
inferences about main ideas and supporting ideas. They should be able to express a 
judgment about the text and about text features and support the judgment with evi- 
dence. They should be able to identify the most likely cause given an effect, explain an 


author’s point of view, and compare ideas across two texts. 
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NAEP Reading Achievement Levels—Grade 8 


NAEP 
Basic 


(243) 


NAEP 
Proficient 
(281) 
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Eighth-grade students performing at the Basic level should be able to locate infor- 
mation; identify statements of main idea, theme, or author’s purpose; and make 
simple inferences from texts. They should be able to interpret the meaning ofa 
word as it is used in the text. Students performing at this level should also be able to 


state judgments and give some support about content and presentation of content. 


When reading literary texts such as fiction, poetry, and literary nonfiction, eighth-grade 
students performing at the Basic level should recognize major themes and be able to 
identify, describe, and make simple inferences about setting and about character motiva- 
tions, traits, and experiences. They should be able to state and provide some support for 


judgments about the way an author presents content and about character motivation. 


When reading informational texts such as exposition and argumentation, eighth- 
grade students performing at the Basic level should be able to recognize inferences 
based on main ideas and supporting details. They should be able to locate and provide 
relevant facts to construct general statements about information from the text. Stu- 
dents should be able to provide some support for judgments about the way informa- 


tion is presented. 


Eighth-grade students performing at the Proficient level should be able to pro- 
vide relevant information and summarize main ideas and themes. They should 

be able to make and support inferences about a text, connect parts of a text, and 
analyze text features. Students performing at this level should also be able to fully 


substantiate judgments about content and presentation of content. 


When reading literary texts such as fiction, poetry, and literary nonfiction, eighth- 
grade students performing at the Proficient level should be able to make and support a 
connection between characters from two parts of a text. They should be able to recog- 
nize character actions and infer and support character feelings. Students performing 
at this level should be able to provide and support judgments about character motiva- 


tion across texts. They should be able to identify how figurative language is used. 


When reading informational texts such as exposition and argumentation, eighth- 
grade students performing at the Proficient level should be able to locate and provide 
facts and relevant information that support a main idea or purpose, interpret causal 
relations, provide and support a judgment about the author’s argument or stance, and 


recognize rhetorical devices. 


NAEP 
Advanced 
(323) 


Eighth-grade students performing at the Advanced level should be able to make 
connections within and across texts and to explain causal relations. They should 
be able to evaluate and justify the strength of supporting evidence and the qual- 
ity of an author’s presentation. Students performing at the advanced level also 
should be able to manage the processing demands of analysis and evaluation by 
stating, explaining, and justifying. 

When reading literary texts such as fiction, literary nonfiction, and poetry, eighth- 
grade students performing at the Advanced level should be able to explain the effects 
of narrative events. Within or across text, they should be able to make thematic con- 


nections and make inferences about character feelings, motivations, and experiences. 


When reading informational texts such as exposition and argumentation, eighth- 
grade students performing at the Advanced level should be able to infer and explain 

a variety of connections that are intratextual (such as the relation between specific 
information and the main idea) or intertextual (such as the relation of ideas across 
expository and argument text). Within and across texts, students should be able to 
state and justify judgments about text features, choice of content, and the author’s use 


of evidence and rhetorical devices. 


NAEP Reading Achievement Levels—Grade 12 


NAEP 
Basic 


(265) 


Twelfth-grade students performing at the Basic level should be able to identify 
elements of meaning and form and relate them to the overall meaning of the text. 
They should be able to make inferences, develop interpretations, make connec- 
tions between texts, and draw conclusions; and they should be able to provide 
some support for each. They should be able to interpret the meaning of a word as 


it is used in the text. 


When reading literary texts such as fiction, literary nonfiction, and poetry, 12th-grade 
students performing at the Basic level should be able to describe essential literary ele- 
ments such as character, narration, setting, and theme; provide examples to illustrate 
how an author uses a story element for a specific effect; and provide interpretations of 


figurative language. 


When reading informational texts such as exposition, argumentation, and docu- 
ments, 12th-grade students performing at the Basic level should be able to identify the 
organization of a text, make connections between ideas in two different texts, locate 
relevant information in a document, and provide some explanation for why the infor- 


mation is included. 
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NAEP 
Proficient 
(302) 


NAEP 
Advanced 
(346) 
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Twelfth-grade students performing at the Proficient level should be able to locate 
and integrate information using sophisticated analyses of the meaning and form 
of the text. These students should be able to provide specific text support for 


inferences, interpretative statements, and comparisons within and across texts. 


When reading literary texts such as fiction, literary nonfiction, and poetry, 12th-grade 
students performing at the Proficient level should be able to explain a theme and 
integrate information from across a text to describe or explain character motivations, 
actions, thoughts, or feelings. They should be able to provide a description of settings, 
events, or character and connect the description to the larger theme of a text. Students 
performing at this level should be able to make and compare generalizations about 


different characters’ perspectives within and across texts. 


When reading informational texts including exposition, argumentation, and docu- 
ments, 12th-grade students performing at the Proficient level should be able to inte- 
grate and interpret texts to provide main ideas with general support from the text. 
They should be able to evaluate texts by forming judgments about an author’s perspec- 
tive, about the relative strength of claims, and about the effectiveness of organizational 
elements or structures. Students performing at this level should be able to understand 
an author’s intent and evaluate the effectiveness of arguments within and across texts. 
They should also be able to comprehend detailed documents to locate relevant infor- 


mation needed for specified purposes. 


Twelfth-grade students performing at the Advanced level should be able to ana- 
lyze both the meaning and the form of the text and provide complete, explicit, 
and precise text support for their analyses with specific examples. They should be 
able to read across multiple texts for a variety of purposes, analyzing and evaluat- 
ing them individually and as a set. 


When reading literary texts such as fiction, poetry, and literary nonfiction, 12th-grade 
students performing at the Advanced level should be able to analyze and evaluate 

how an author uses literary devices, such as sarcasm or irony, to enhance and convey 
meaning. They should be able to determine themes and explain thematic connections 


across texts. 


When reading informational texts, 12th-grade students performing at the Advanced 
level should be able to recognize, use, and evaluate expository and argument text 
structures and the organization of documents. They should be able to critique and 
evaluate arguments and counterarguments within and between texts, and substantiate 
analyses with full and precise evidence from the text. They should be able to identify 


and integrate essential information within and across documents. 


APPENDIX C 


SPECIAL STUDIES: NAEP READING FRAMEWORK 


Three special studies were proposed as part of the development of the 2009 NAEP Reading 
Framework. Although very different in topic, they have the common goals of improving the quality 
of the NAEP assessment and gaining maximum information about student achievement in reading. 
One of the special studies (meaning vocabulary) can inform test development by providing informa- 
tion about new item types. Other studies propose using data gained from the assessment to examine 
English learners’ reading achievement as well as factors that have an impact on the gender gap in 
reading. Further details about the special studies, including methodology, appear in the 2009 speci- 


fications document. The special studies are presented here in priority order from highest to lowest. 


Developmental Study: Meaning Vocabulary Assessment 


Purpose 


Looking toward the addition of meaning vocabulary items to the NAEP Reading Assessment, 
this developmental study will evaluate the reliability and the construct, content, criterion, and 
concurrent validity of the proposed method of measuring meaning vocabulary. The study was 
conducted in advance of the 2009 administration of the NAEP Reading Assessment to inform the 


development and use of meaning vocabulary items on NAEP. 


Rationale 


Although NAEP has included a few vocabulary test items in the context of passages on past 
assessments, the number of items was scant, and there were no specific vocabulary criteria for 
selecting the items or distractors. Furthermore, past reports from NAEP provided little information 
on how students performed on the vocabulary items and whether that performance was associ- 
ated with comprehension achievement levels; thus, these reports did not provide a foundation for 
emphasizing the importance of vocabulary to reading comprehension. The importance of vocab- 
ulary in reading comprehension, as supported by research, will be much more widely understood 


and disseminated with NAEP’s initiative specifying vocabulary as a major component of reading 
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comprehension; NAEP reports providing quantitative data about the performance of 4th, 8th, and 
12th grade students on meaning vocabulary questions and the developmental differences among 
grades; and NAEP reports describing the differences among Advanced, Proficient, Basic, and below 


Basic readers on vocabulary and the implications of these differences. 


Recognizing a growing body of research that supports the argument that vocabulary is crucial to 
reading comprehension, the NAEP Reading Assessment includes a measure of vocabulary. All vocab- 
ulary items are to function both as a measure of comprehension of the passage in which the word is 
included and as a test of readers’ specific knowledge of the word’s meaning as intended by the author. 
NAEP will include a sufficient number of items to provide reliable and valid data for analysis and 
interpretation. A description of the criteria for word selection and number of items appears in chapter 
two of the NAEP Reading Framework and will be elaborated in the specifications document. 


Research Questions 


What is the correlation between reading comprehension and meaning vocabulary items, and 
how does the addition of meaning vocabulary items affect overall scores on the NAEP Reading 


Assessment? 


How does the introduction of meaning vocabulary items affect the scores of ethnically, socio- 


economically, and geographically varying groups and low-, average-, and high-performing readers? 


What is the correlation between scores on the meaning vocabulary items and a vocabulary test 
such as the Peabody Picture Vocabulary Test, 3rd Edition (PPVT-III)? Answers to this question will 


address the concurrent validity of NAEP’s vocabulary measure. 


Special Study: English Language Learners 


Purpose 


This special study will examine the patterns of achievement among English language learner 
(ELL) students and the link between NAEP scores and other indicators of students’ ability and 


achievement, as well as the effects of the accommodations afforded students in these groups. 
Rationale 


In today’s schools, the number of ELL students is on the rise. This population trend has impli- 


cations for reading instruction and assessment as educators seek better ways to teach and evaluate. 
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Clearly, they need more information about language and its relationship to reading comprehension 


and meaning vocabulary, a link indicated by past studies. 


Although past NAEP reports have provided scores by ethnicity, they have not provided infor- 
mation about the link between language minority students and reading ability. This special study 
seeks to examine this link, informing the discussion of how to develop a dynamic assessment (adap- 


tive testing) that more accurately maps the achievement of U.S. students.” 


Research Questions 


What miscues occur most frequently among different ELL groups, and are these miscues consis- 


tent with different groups of English learners’ speech? 


Are tests of English language proficiency predictive of NAEP comprehension and vocabulary 


scores? 
What are the differential effects of English proficiency level on NAEP reading and vocabulary? 
How are reclassified fluent English proficient students (RFEP) achieving in comparison to other 
groups in reading comprehension and vocabulary, and how do they progress after one, two, or three 


years of reclassification? 


At what minimum level of English proficiency is a student able to take a NAEP Reading Assess- 


ment written in English? 
Do accommodations given to ELL students give access to or change the construct of the test? 
This study was not conducted because of lack of funding. 


Special Study: Gender Differences 


Purpose 


This special study examines the differences in reading achievement between boys and girls, 


focusing on factors associated with the gender gap in reading. 


” The ELL special study may be informed by the results of the National Literacy Panel's study on language minority 
children and youth. The NLP conducted a comprehensive review of research on the development of literacy among 
language minority children and youth that was completed in 2004. 
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Rationale 


The gender gap—a significant difference between the performance or achievement of boys ver- 
sus gitls—exists in a number of education-related settings and situations. Girls generally have higher 
secondary school graduation rates, college admission rates, and enrollment in Advanced Placement 
courses in the humanities, whereas boys have a higher incidence of diagnosed reading disorders. 
Although boys generally have higher mathematics and science achievement, the gender gap in the 
language arts favors girls. Results from the 2002 NAEP Reading Assessment indicate the following: 


e The score gap between male and female grade 4 students in 2002 was smaller than in 2000, 
but it was not found to be significantly different from that in 1992. 

e The score gap between boys and girls at grade 8 was smaller in 2002 than in all prior assess- 
ment years. 

e The score gap between grade 12 boys and girls in 2002 was greater than it was in 1992. 

e Girls outperformed boys at all three grades in 2002. 


As educators continue to grapple with the gender gap’s implications for instruction and assess- 
ment, this special study will examine variables in NAEP’s assessment design and their relationship 
to the gender gap in reading. This study will look specifically at the NAEP assessment design and at 


achievement data gathered from the 2009 administration. 


Research Questions 


How are question response modes (e.g., multiple choice, constructed response) related to 


reading achievement? 
How are the types of texts (e.g., narrative, information, poetry) related to reading achievement? 


How is the content of the selection (e.g., gender of main character, different themes, presence of 


moral) related to reading achievement? 


This study was not conducted because of lack of funding. 


| 72 


73 | 


| 74 


75 | 


“ae NATIONAL ASSESSMENT GOVERNING BOARD 
~~ U.S. Department of Education 


SLOLOM\TeDadiMOrleVi Ce) MSSeacILeM AW AESIUILCMsY2oM | EVSaTTIN:400) Ma DL Onr401010 am mR AANA T-Vel ome{0)"] 


